I know that in Photoshop when I enlarge something, what was once a jagged coastline now becomes blurry. So what I do is make my base map in black and white and then use the pixellate > crystallize filter then bump the contrast all the way up to max, this takes all of the lil crystals that appear in the blurry part and makes them black or white. This, then, restores a jagged coastline but It doesn't quite line up with the old one, which is usually not a big deal. So look through and see if you have something similar to pixellate or crystallize in GIMP.