VS 2010 will open a project for the first time a little slowly because it will cache all the header files used by your sources, but overall compilation times should not suffer nor should the runtime debugger. Subsequent builds should be faster if you have the minimal compile flag set (under C++/Code Generation), which it is by default.
You might want to check your box to make sure all the hardware is running optimally. If after closing VS you're still having slowdowns, then it could be a hardware problem. Check your CPU and motherboard temps, check your RAM for memory faults and excess paging by the OS, check your HD fragmentation level and clean it up, and test your video card.
I'm not a big fan of XCode. I would personally use Eclipse over it simply because the UI in Eclipse is better. GDB is also better integrated into Eclipse compared to XCode.
i'd like to skip this M\\$ crap for ever
Some things MS does is undeniably bad, but Visual Studio is definitely not one of them. No IDE comes even close to productivity.