You render a front depth map, at full game res.
then you render back faces, half or quarter res, and side views, and top and bottom view.
then you make "keep maximum" mips of all the depth maps.
the keep maximum mips help you traverse the depth maps without accessing all the pixels at once, leaving you more cache coherency.
You could then get a full bounce of gi by tracing rays through the displacement maps, at real time.
I was wondering why this hasnt been done yet? Its got to work!
its hybrid... rastering to make the displacement maps, and raytracing for the global illumination, even possibly reflections.