No need to abandon. Yes indeed the caves 'leak' into the background - there are one or more gaps in your lines.

I was easily able to isolate the background in gimp. Here is the shaded map. I also attack a black .png with just the isolated layer so you can select it.

Click image for larger version. 

Name:	Map.png 
Views:	555 
Size:	122.2 KB 
ID:	51789

The method I used was: select by colour (all black). Then stroke selection 2 pixels with a black pen/ or else enlarge selection by 1 pixel and fill with black.

Click image for larger version. 

Name:	MapBlack.png 
Views:	253 
Size:	33.4 KB 
ID:	51790

Note: I did a quick job so there are errors that would need to be touched up.