You might want to think about that perfect town icon you are using because it matches the others on the map. If it's supposed to be a "mashup of architecture" you should consider making another icon that is not so clean cut. As in, have the walls be slightly off and such. Not sure how you can do that but I think if the icon of the town is slightly different it might help. It almost looks like you have blurred the other towns a tad and left Mosford sharp. Maybe you should try it the opposite (blur Mosford and have the others sharp).
You have a nice map going here but it sort of looks more like it was picked up out of the dirt rather than indicating that just Mosford is dirty.
One other thought I had, not sure if it will work, but you might add a gradient of darkness along the rivers edges and have it expand into Mosford. My thought is that maybe it would give the idea that Mosford is a 'muddy' town.
Just be sure to save your original layers while you experiment.
Hope this helps or at least gave you something to think about.