Well, some of those would be feasible in stone - if they were mostly solid stone. Think Japanese castles, which are mostly a massive stone block wall backed with rubble and fill, and a wood-and-plaster fairytale castle built on the resulting platform(s). A lot of European gunpowder-era forts have the same principle. So your fantasy fortress would be mainly solid for most of the lower portion, with the exception of a few staircases and tunnels, perhaps.
They only sink into the ground if there is nothing solid under the foundations, you would want to build all of those on solid bedrock.