Ive got this qdm (quadtree raytracing) implementation that works on 6 displacement maps (one for each face direction) inside a bounding box. (with instancing)
I could instance as many bounding boxes as id want, (as they are only 8 points each) and raytrace the insides, only problem is - you cant have them penetrating each other because there is no way to detect inner intersections as they are all raytraced independantly.
Is there any way around this problem? or is it just simply impossible.
hmm, I kinda thought of a solution, you have to draw the overlapped boxes after each other in separate passes... hmm, maybe you can do it.
If your pixel shader outputs a depth value in addition to a color, then depth testing should clean up the intersecting objects, just like it does with plain old polygons.
In my instancing raytracer I keep sorted list of events where current ray hits bounding boxes (4-8 items). When do I detect intersection with solid geometry I check that list and if there are bounding geometry before intersection I'd continue tracing.