I might have left it a bit late to weigh into this argument, but one consideration: does your town need defences at all? Almost every medieval map on these forums is given a wall, but this is not historically realistic. If this town isn't actually facing any threats, it doesn't need a wall. And as for Vellum's idea about the overseer's office - it would be useful to be able to watch the workers from high up, but you also need to be close to the workers to resolve problems. Imagine having to trek all the way up the cliff every time you break a pickaxe handle.