My star chart map is 7500x5000 and that has around 300 layers. It does run rather slow, when zooming and such, but its not unbearable. And my machine is about 3 years old (Vista/AMD 4400/4GB.
I've recently upgraded to PS CS5 and it definitely runs smoother than previous versions.