Update:

I have been crunching some numbers regarding the size of this small town (which is nestled near the wall, not an actual walled town), and the garrison needed to man the towers along this 7 mile long wall (8 towers in total), is going to be 400 men (I am figuring half a platoon per tower, swapped out every couple of weeks, that is a full company on the wall, with a second company in reserve/manning the gates, plus ancillary headquarters troops).

And figuring in the sort of "town-based" support such a troop (not to mention travelers/adventurers would need, and the town is now about 1000 permanent inhabitants (well, about 850 since about half the garrison is on the wall at any given time)...