It's pretty much as you say: if you use Gimp or Inkscape, you have more 'freedom' about the look of what you create, but with symbol driven software (e.g. Campaign Cartographer) you are limited (until you learn to make your own symbols) to the symbols they give you - also these programmes don't give you so many layer style options etc.
No one tool is 'best' for mapping. Many people use more than one item of software to make maps. What you have to find is the one that's best for you, and that usually involves trying out a few (many programmes you pay for have evaluation versions), or by looking at completed maps, seeing what you like and what the author used to draw them with.
Best of luck in your search!