Depends entirely on the style of the map. For pure fantasy or sci-fi maps with small historic relevance, something that connects to the setting is usually a good choice, like dwarven or elven runes, some pattern that could be drawn by one of the cultures in the area, etc. It should probably be kept as simple as possible though, to not distract from the map itself. If you're making something history-based, the best way is to look at real maps from the period and draw something inspired by that (or even download a map and use the very same border, if the copyright for the chosen map has expired). For modern maps, I think something extremely simple looks best; a single or double line in the same color as the graticule, for example.