Look at the open source engines source for some scene graph and scene managing implementation (OpenSceneGraph, Ogre, CrystalSpace, ..).
Basically you want something called SceneGraph, in SceneGraph you have Nodes, attached to each other. You attach anything renderable to the nodes. When you move node, every subnode and attached renderables move too. Same applies to rotation or any other transformation.
When you outline this, you can try to implement a Octree optimization. Octree will build octane tree using attached renderables.
Physics is a little tricky, because the standard culling techniques aren't applicable to physics. I solve it this way:
1) using std techniques to render (Octrees)
2) using physics scene optimizations parallel (got BruteForce - no culling, and basics of some sort of tree - scene divided into little boxes, collisions only happens from one body to bodies in neighbour boxes)
btw: I know I can collide using octree, however I think this approach is much more flexible than constraining physics to rendering scene managers ...
@karligula: I agree, collision and physics is fun, but only when it has been already implemented and you're watching the result