On reflection, looking at the three views, I really like the isometric view - a lot! I'm thinking of keeping everything isometric, that way I can create the rest of the castle grounds, ancilliary buildings, moats, walls, gates, etc. Create the island. Put the castle on high ground, that drops to each bailley, reaching a narrow channel cut on the down stream side of the isle, really making two very close islands separating the fortress ground by a natural moat between the town part of the larger island upstream with a lower elevation.
Because some of the streets will be hidden from the angles behind buildings, the gap between should be noticable, I might even add street labels and labels to specifc structures in the castle town, notable retainers, temples, the floating world district, etc.
I'll do separate layers of every stage, cut away the excess, create a 3D island, create districts of town separately, but with multiple buildings close together with surrounding walls, then composite the final in Xara. That's the plan anyway. We'll see if it works!