Thank you all for your help, guys! Sorry for the slow response, I haven't had the mental capacity for trouble-shooting in the last few days.
No, but I drew another line at an angle after increasing resolution and it didn't make a difference to the jaggedness. I did try with a black outer glow set to Normal at 100% and size 2px, but it's a lot fainter than the stroke. It would suffice if I can't fix the stroke issue, though, so I appreciate the suggestion. I never thought of using outer glow for stroke before. (I'm not sure how to do the shape burst thing, though.)
Sadly, adjusting the brush spacing to 1% didn't seem to do anything for me.
I normally use the pen tool, then right click and "stroke path" to get the main line, then use layer effects for the stroke. In this case, I'm stroking the path in white (10px for the city walls and major roads) and adding a 2px black stroke in blending options.
I can get much smoother lines using the Edit > Stroke... menu instead, which was a suggestion that came up when I went looking for answers on Google, but it's a more destructive option because it's not editable like layer effects are. (You're right that the usual Pen > Stroke Path option uses the current brush settings where Edit > Stroke doesn't seem to. I've tried fiddling with the brush, spacing it more or less and changing the softness, but it doesn't seem to the affect the stroke via blending options.)
It's the outline - the black stroke added by layer effects, and not the Pen > Stroke Path - that's the issue. The actual stroked lines come out just fine, but if you add the stroke via blending options, you get a jagged line.
The thing is, I never had this problem before. Stroke on layer effects was always smooth. Unfortunately, the only vector programme I have is Paint Tool SAI, and it's not great for straight lines. This is for a request due in January so I don't want to have to learn a new software like Inkscape when I should be working on the map. (Then again... the time it takes to fix this issue might be worth the leaning curve!)
The problem with the zoom is that the thumbnail is the smaller version. I took the screenshot at 100% zoom (CTRL+1) so the image you click on is the actual size. If you're looking at full view, you get the jagged black lines, which shouldn't (and never used to) happen.
Ooh, I see! This is for print, so I should have started at 300 anyway, but it's good to know it doesn't affect rendering like that.
I'm not sure what the thumbnail is, but the screenshot is of full 100% zoom (CTRL+1 in Photoshop).
The square towers were created square then rotated, yeah. The circles were not rotated and still have the jagged stroke on some of them. I've got image interpolation set to bicubic (default) if that changes anything.
The image the thumbnail leads to is a screenshot of the map at 100% and it still shows jagged edges. The thumbnail itself looks okay, though. I'm not sure what size I'd need to make the map for it to look good at print size. :/
I tried adding a glow and stroke via layer effects to a 10px line. It does look better, but the glow needs to be 10px as well to hide the jagged stroke, which just doesn't look right on this map (especially not also applied to buildings).
How big would you suggest I make the map for printing in a book and also hide the problem of jagged lines? (The author hasn't specified how large it actually needs to be despite asking a few times so I can't give more information than that. I'd guess it's for a regular hardback, though.) I started out with a 2000x2000 canvas at 150ppi with intent to sketch the layout, but I'm used to creating for digital only so I have no idea what size to up it to now.
Thanks again for all your help and suggestions. I really appreciate it!