I will do my best to clear this up for people as it greatly concerned me as well. I was using Vista 32bit with 2Gb of ram. My system was sharing 756Mb of my ram to Video and with some lazy research I found that 64bit Vista would allow for more ram to be used...so I upgraded to 64bit and bought another 2Gb of ram (4Gb total). Now my shared memory is 1780Mb. I nearly threw the PC out of the window. Well, forums all over say a million different things so I swallowed my pride and readied my Visa and called Microsoft Technical Support. (it ended up not costing me)
Vista does share a large chunk of your system memory for Video Purposes. However, Vista does not work the same way we are used to with onboard video solutions. Shared memory in Vista does not mean the memory is gone purely to video. In fact, it is hardly ever used for video purposes unless you do a lot of modeling, rendering, or play games at extremely high settings. (or just have a very low end video card) The memory is primarily still used for every day computer useage which is why you do not notice any slowdown on your system, and if you have, it's highly unlikely that it is because your memory is being shared. I wish I could explain it better but if this doesn't ease your concerns you can always call them...they seem to love talking about how wonderful they're memory design is.