Yeh I'm not set on any details yet, the above was just a quick 1hr sketch while I drafted the idea, but I definitely see where you are coming from. Thinking a little more on it I'm going to try and design the castle out of dragon parts (smaller dragons) like the walls are made from scales/dragon hide and the classical conical tower rooftops made of dragon wings. I think given the size I wont be able to add all that as detail, but maybe imply the textures and details and then do a magnified section of one of the towers to show the scales.

Name:  dragon towers.jpg
Views: 617
Size:  66.8 KB

Thinking maybe these guys dropped the initial big dragon, but now they're dragon hunters, each time they kill a dragon they build a new tower from its body (and there's going to be dozens of towers on the castle).