I think I largely agree with you. The objective I aim for in my games is to create an environment that my players are happy to immerse themselves in. Any element that threatens the "willful suspension of disbelief" (a possibly overused phrase by this time) needs to be examined carefully.
Although excessive attention to realism tends to bog down the cartographer so that maps are never completed, I think there is a more insidious danger. Fantasy cartography is all about the fantasy, and fantasy deals routinely with things that are most emphatically not realistic. Attempts to make a fantasy world conform to the most minute detail of real geology, etc sap the joy from playing from me. That's the reason I am not terribly interested in Paul Cardwell's Mythworld. Heroic fantasy characters shouldn't have to worry about getting infections from the minor cuts they receive while fighting skeletons in swamp water--unless the Evil GM wants to use such an infection as the basis for the next adventure, of course.
I guess you already said the same thing in a much pithier fashion, but I was feeling compulsive this morning.