Step 6 - if you already have a pattern of evenly spaced hand drawn lines, awesome! if not, you can make one. Without going into a full tutorial, create a new document, make sure it's square (256x256 should be plenty for most purposes, though you can double it if you REALLY want to avoid possible tiling). Draw lines straight across (just freehand them for the best look, play with jitter and scatter if you want to hold shift to draw them out straight) every 5 or 10 pixels, mess around to make it seamless, and you're good. If you're not up for that, you can cheat and use this as your pattern (512x512)
Step 6.5 - Now the fun part. Change mode to Bitmap and mess with the settings. I find I have to convert to greyscale before I can convert to bitmap (if there's an easier way, I haven't found it yet). Go to your menu bar - Image/Mode/Greyscale (it will ask you to flatten, you can say yes, if you say no it will flatten when switching to bitmap mode anyhow). Now Image/Mode/Bitmap - and here's where the magic happens. Change your DPI to double your current value. I usually work at 300 DPI so I switch to 600, if you left yourself at the screen default of 72, you would put in 144, etc. But wait! there's more! Under Method, tell it to use a custom pattern, and choose that nice set of lines that you set up earlier.
Step 7 - You should now have a rather large image of white background with a lot of straight choppy looking lines that dither off to nothingness at the ends. Awesome. It should look something like this...
Step 8 - Now for the second half of the magic - bringing this back into the original file. But wait, if you move it across as is, you'll have a layer that's 2x the size of your actual map, and if you shrink it now, you'll have a few minor headaches later on. So we'll work it over first. convert back to RGB with the menu, Image/Mode/RGB. Select the black with your magic wand tool - I turn off the antialiasing for this, and ensure Contiguous is turned OFF. Now that my black is selected, I copy that to a new layer. Then I resize the image - the quick and dirty way to do this is just use the DPI value, put in the same DPI of the original file (300, or 72, or whatever you started with). This will shrink it to 50% and match your original file again.
Step 9 - Duplicate your layer, and move it back to the original file. Move the new layer so it is underneath your land outline layer. It should look like this, if you left your land shape layer as white:
If you don't want to use your Land Shape layer, you could also select the land shape layer (ctrl click the thumbnail) and put a mask over the wavy line layer.
Continued in next post due to image count per post limitations.