There isn't a hard limit in the Fractal Terrains source code, but the program reaches a practical limit beyond which Windows refuses to allocate the image. I vaguely recall it that it will tend to top out around 8000x8000 pixels or thereabouts. That works out to about 256MB of image data, so that's really not too bad on a 32-bit program that can only address 2 GB of memory at the best of times.
Note that playing with very large images in FT tends to cause crashes as it runs out of memory. It's not a terribly well-behaved program with regard to how much memory it allocates. I have been informed that efforts are being made to reduce this problem for future versions, however.