That's a pretty big list. If he's looking to retire, he might want to keep a small staff instead of a large one. You have quite the suite there. My inclination whenever I need a castle is to steal a floorplan from a real castle and see if anyone notices. I think the result of what your describing could fit a number of basic designs, including the one above. But when building a setting like this, the most important question (IMO) is - is it going to matter? If the place is going to become a ruined adventure site, then yes it's important to know where the walls & buildings are. If, on the other hand, it is the home of a questgiver or mentor, a simple description - "a small rectangular castle with a single tower three stories tall" for example - ought to suffice.
My instinct, based on your original description, would be to use this:Restormel Castle. Convert the chapel into a wizard's tower, remember that the other buildings are 2-stories, and you're done. Bonus - you have images to show your players!