I'd like to ask if anyone has some experience with shadow volumes and recent stuff. Right now I'm working on implementation (I need them just for few tests right now, but I wonder whether it's worth it to use them "in final product").
Of course I mean mostly the performance. And maybe some hints on overcoming their problems (Alpha-tested geometry and stenciled shadow volumes? Transparency and shadow volumes? Are there any usable solutions these days - ofc I could project the transparent surfaces as texture (but well then I could store shadow map in one channel and do shadow mapping too)).
So, I've finally found some times to do experiments with geometry shaders. I've only tested it on notebook GPU - where it performs ... well - at least for notebook GPU. I can run sponza with stencil shadow volumes (single light).
On desktop I added some static objects to get around like 1M triangles and it still ran fine (at maximal fps). Then I got bored of adding objects .
So basically I can get sharp shadows using shadow volumes right now. Soft ones can be done through post-processing right now. I'm also going to try penumbra wedges some other day too. I though don't think I'll be able to optimize these further (of course apart of using level-of-detail techniques and such), and I just don't want to accept using lower resolution meshes for shadows.
So far we need to try this in some more serious game environment - so that we'll be able to decide whether we're going to use shadow volumes or not (or at least give user the option to turn them on).
And at last few words why we're working on shadow volumes right now - we would like to achieve more real soft shadows (like penumbra wedges give), because PCSS and such techniques on shadow maps have so many artifacts (while shadow volumes don't). The most problematic part here is performance, because the volumes are quite heavy for fillrate (and our scenes are already fillrate-heavy, so we've got some heavy optimizations running there (like frustum & occlusion culling inside BVHs, etc.).
The second thing is shadow map limitation for scene size. This problem although isn't that big one, simply because we can use cascaded shadow maps. But there are problems with micro-geometry not casting shadows (we'd of course like to show shadows for micro-geometry - rendered only for near objects of course).
I hope to write again in few days with some more useful scene (and pictures finally) + running penumbra wedge, and some numbers on performance.
the other alternative is shadow maps, i always thought volumes were a better approach... although id just be doing the basic hard edged variety. I sorta made a promise never to go back to nasty limited shadow maps id rather raytrace than go back to them.
The problem is, that except of ray tracing both solutions have quite ugly disadvantages.
Shadow maps work pretty well independently of scene geometry (note that not totally, because when your scene has 10M tris, you actually have to render 10M tris to render target), but it doesn't generate any new geometry. They're texture based (so alpha-tested geometry friendly), so basically performing filters is simple, yet introducing aliasing (which is bad and insolvable issue - also the more we try to get rid of it, the over-complicated our shadow rendering code gets). They also have other severe artifacts - biasing troubles and self shadowing (which looks really ugly in some cases - and you most likely will have it in your game).
Shadow volumes on the other hand are generating lots of new geometry (this isn't so big problem as we have tessellation & geometry shaders these days), stenciling is fast enough (same goes for visibility buffer computation for penumbra wedges). Filtering can be done through penumbra wedges (although this introduces even more geometry, but complexity of code is just slightly increased). The problem is that your models needs to store triangle adjacency information (twice more data actually), and adjacency computation isn't something really fast - so you want to precompute it (and sometimes indices approaches just doesn't work well - so you have to come up with more robust (and slower) solution) - of course as you can pre-compute this it's not problem anymore. Self shadowing is fine, but they need tessellated geometry to work well on angles (this pretty much can be solved with tesellation shaders, or introducing bias (which is actually never good way for really robust solution).
Ray traced shadows are only one perfect solution. On the other hand performing shadow rays cast can be quite expensive (even in compute shaders) and if you have performance to do that, you want to use the ray casts to other stuff (reflections, GI, etc.).
Currently works like a charm. Pixel perfect Z-fail shadow volumes.
As for performance, it works quite well (extrusion is done in geometry shader, using adjacency triangles), I can render \~60k triangle scene on my notebook graphics (on desktop it's more like 2M triangle scene).
Wedges are still work in progress (the thing is, that wedges geometry shader works, but I still haven't figured out how to correctly use visibility buffer to get proper shadows - will elaborate further on this).
EDIT: For shader on simple Z-Fail shadows from point light - http://pastebin.com/YaCi7azQ