I wouldn't say this looks amateurish, I'd say it looks like a preliminary map! I like the shapes & placement of the landmasses, although the coastlines seem a bit over-scraggy (sorry 'bout the lack of better language). I know smooth coastlines are a bad idea, but yours just seem too rough.
For your mountains, I woud suggest using a rough brush (if you want, I could upload a couple I like using) on a very low opacity setting (not in the brushes panel - see along the top of Photoshop, there's an option to change opacity and flow). Just click multiple (okay, a dang lot of) times in the mountain areas to build up your mountains. Incidentally, I'm kinda talking based on the idea you'd use something like Pasis' terrain tutorial like I did ^.^ If you're not, it's still a viable option; you could also manipulate DEM files in photoshop to build up your map - this would be a lot of work and would be more satellite style, so I dunno what you want this map to look like in the end. Oh, and I also like the placement of your mountains - dunno if you figured out where they'd "really" be based on totally analytic "real-world" geographic stuff (like tectonics) but they appear to be pretty well placed. Good luck!
- Alizarine