The new mountains are great in terms of the texture but I'm going to agree with feanaaro - they like an L - the angle at the bottom is just too sharp. Just soften that out into a curve and it'll look a lot better.

As to your coastlines - I'm not enough of a geologist to comment on realism of them overall, what I can do is offer you a trick that I use to provide a bit more randomness to coastlines once you have a shape you like. First of all you need a brush that isn't just a solid shape. Pretty much any brush can work - but a spraypaint like one that is a mass of dots works well. Then what you need to do is play with the brush tab on the brush window (f5) - particularly the scattering and shape dynamics sections. You want to get something with a fairly solid core to it, but with randomised protrusions from that core - something like this:
RandomBrush.png

Next get a selection of your entire landmass and convert that selection into a path using the 'Make work path from selection' option on tha paths tab (Windows - Paths if you don't have it visible). And then right click on the path and select stroke path. This allows you to stroke a path with the selected brush (rather than just a uniform stroke) so allows you to generate some randomness on your lines. Obviously you'll want to play around with the size of the stroke and the qualities of the brush to get a good effect - depending on the size of your map. Hope it helps, let me know if anything isn't clear.