I think I'd favor keeping the steward and head maid's chambers in the hall, but set aside a small chamber adjacent to the lord's chamber, and perhaps one for each guestroom, for a valet. Also, perhaps there might be a joint office for steward and housekeeper, so the business of the keep does not interfere with the wizard's experiments and/or leisure. If it weren't a wizard, I'd suggest that sort of thing could be taken care of in the library.

There are likely to be semi-permanent structures built along the inside of the perimeter wall for miscellaneous castle work: a farrier/blacksmith, candlemaker, cooper/carpenter, etc. How many such people might work in the castle is dependent on the distance to that town and how much of its services the castle can make use of. The crafters who work these jobs may have quarters in their shops, or they may be housed outside the walls.

Don't forget a cistern or well, garderobes, and cesspool (and possibly a gong farmer to care for it).