It depends, I saw the castle more as a coastal outlook fort than a central capital city type capital. (Basically 1 step up from a look out tower)

I don't think the black outlining works on the battlements either, I think removing this may help it match the feel of the background.

Also what is the thing in the north?

EDIT: Nevermind I got the terms mixed up, I think the harbour fortress is fine, I agree the main castle seems a bit small (mainly the central tower bit of it)