In a fantasy setting, I'm of the mind that the villages, towns, and cities should be whatever size best fits with your own image. Historically, the population was spread out all over europe in small farms and villages that stretched between the large cities. If the lands are full of monsters, especially organized creatures such as orcs or goblins, spreading out like that in a fantasy world would just provide easy pickings. I would expect all the farms to be clustered as close as possible around the cities for protection. Also, in a world of fantasy, there may well be crops and livestock that can be grown/bred more efficiently and in much smaller areas. Magical aid may be used to assist in feeding the populace as well (protection against insects and weather, etc).