Yeah ctrl+click the layer with the glow, that's just for the outer glow or stroke issues. Otherwise, just doing that to your land mass seems to work pretty well too, it doesn't really smooth it out or destroy the cragy-ness of it . It gets rid of stray pixels that are slightly opaque or helps shore up the lines basically. That only works if you have a layer that's the large mass. Which one is Tear's Tutorial for mountains? I probably looked at it, I just don't recall which one it is.

Of course, take my stuff with a grain of salt too, not like I'm a professional.