You could use the same techniques as they use in CityEngine.
It basically works by using a mix of supplied 2d bitmaps to define parameters, and a library of objects to fit into the map.
You can then build your own rules for plant viability.
So say you have a building with a mud floor, that has a high viability. A concrete floor has low viability. A park has a viability of 1, tarmac a viability of 0.001
Then you can run a simple algo that generates a 2d bitmap of initial plant growth. Different plant types can exist in different viability levels.
Each plant type has a damage value, this is the amount of damage the plant can do to the fabric of the city. This increases the viability of regions next to the plant.
Iterate this a few times till you have the result you want.
Then use the final viability table to do two things.
1) Damage the buildings, bring down walls, smash windows, etc.
2) Add 3d plant meshes
From the viability table you can tell what meshes to add where, I would probably keep another bitmap as well which holds plant age so you can use that to define the size of the meshes. Small tree, large tree, that sort of thing.