Thank you all for your help, guys! Sorry for the slow response, I haven't had the mental capacity for trouble-shooting in the last few days.

Quote Originally Posted by Meshon View Post
Did you draw the layer over again after you increased the resolution? You could also try using the outer glow effect to make the stroke, or use the shape burst option with a gradient stroke to see if that softens up the jaggies. Best of luck!

Update: I just tried it out and adjusting the spacing of the brush to 1% made a little difference. Also drawing the shapes with the pen tool seems to give slightly better results.
No, but I drew another line at an angle after increasing resolution and it didn't make a difference to the jaggedness. I did try with a black outer glow set to Normal at 100% and size 2px, but it's a lot fainter than the stroke. It would suffice if I can't fix the stroke issue, though, so I appreciate the suggestion. I never thought of using outer glow for stroke before. (I'm not sure how to do the shape burst thing, though.)

Sadly, adjusting the brush spacing to 1% didn't seem to do anything for me.

Quote Originally Posted by J.Edward View Post
I agree with Meshon.
It depends how you are getting the 'stroked' line.
You can use edit-stroke to stroke a selection which can be a fast option but it's almost always better to do a pen tool path and stroke it.
There are more option that way. I always do the stroke on it's own layer as well.
When stroking via path I believe you are using the current brush and its settings.
I don't know if that applies with edit - stroke. If stroke via path - do you have smoothing set in your brush settings?
I normally use the pen tool, then right click and "stroke path" to get the main line, then use layer effects for the stroke. In this case, I'm stroking the path in white (10px for the city walls and major roads) and adding a 2px black stroke in blending options.

I can get much smoother lines using the Edit > Stroke... menu instead, which was a suggestion that came up when I went looking for answers on Google, but it's a more destructive option because it's not editable like layer effects are. (You're right that the usual Pen > Stroke Path option uses the current brush settings where Edit > Stroke doesn't seem to. I've tried fiddling with the brush, spacing it more or less and changing the softness, but it doesn't seem to the affect the stroke via blending options.)

It's the outline - the black stroke added by layer effects, and not the Pen > Stroke Path - that's the issue. The actual stroked lines come out just fine, but if you add the stroke via blending options, you get a jagged line.

Quote Originally Posted by Azelor View Post
Make them bigger. At this size, every line with an angle will look pixelated. Even if you try too smooth them, there is not enough pixels to make it look smooth.
Another solution would be to use a vector program. It will always have smooth lines. It's perfect for the line work and then you can import it to photoshop.

Also, it only look jagged when I zoom in, the preview look just fine.
The thing is, I never had this problem before. Stroke on layer effects was always smooth. Unfortunately, the only vector programme I have is Paint Tool SAI, and it's not great for straight lines. This is for a request due in January so I don't want to have to learn a new software like Inkscape when I should be working on the map. (Then again... the time it takes to fix this issue might be worth the leaning curve!)

The problem with the zoom is that the thumbnail is the smaller version. I took the screenshot at 100% zoom (CTRL+1) so the image you click on is the actual size. If you're looking at full view, you get the jagged black lines, which shouldn't (and never used to) happen.

Quote Originally Posted by Midgardsormr View Post
First, a bit of clarification: Changing the print resolution (dpi or ppi) will do nothing regarding what's on your screen. It will only come into play if you print the image out. So you can take that out of the equation straight off.

Are we looking at a 1:1 view here or a zoomed-in screenshot? Were the aliased towers created square first, then rotated, or did you draw them at their final orientation?

If all of the towers are just copies of the "upright" versions and simply rotated, then I can say with relative assurance that the pixelation you are seeing is not an artifact of your stroke but one of a lack of filtering. You can change the default filtering method that Photoshop uses in the Preferences > General. Image Interpolation is the setting to examine. If you have it set to Nearest Neighbor, that would account for the aliasing. Nearest Neighbor doesn't do any color averaging at all, so it will almost always produce the "jaggies."
Ooh, I see! This is for print, so I should have started at 300 anyway, but it's good to know it doesn't affect rendering like that.

I'm not sure what the thumbnail is, but the screenshot is of full 100% zoom (CTRL+1 in Photoshop).

The square towers were created square then rotated, yeah. The circles were not rotated and still have the jagged stroke on some of them. I've got image interpolation set to bicubic (default) if that changes anything.

Quote Originally Posted by Carnifex View Post
Easy! Your screenshot looks just the way it should do. You must increase the size (in pixels) of the image/map and simply draw "bigger" walls and towers!

A pixel image (such as the ones you make in Photoshop ALWAYS look "jagged" when you zoom more than 100% - set the zoom to 100% and it will look good.

(The reason some towers look "sharp" are because they "line up" with the pixels)
The image the thumbnail leads to is a screenshot of the map at 100% and it still shows jagged edges. The thumbnail itself looks okay, though. I'm not sure what size I'd need to make the map for it to look good at print size. :/

Quote Originally Posted by snodsy View Post
Really you want your map sized so that your main line work your using is at least 4pixels. If your using 1 or 2 anything that's not horizontal/vertical will have jagged edges that look pixelated, actually all sizes will have jagged edges but you don't notice as much with thicker lines. Also by putting a glow on them, you make the adjacent pixels closer in color so it looks better, but it's not crisper.
I tried adding a glow and stroke via layer effects to a 10px line. It does look better, but the glow needs to be 10px as well to hide the jagged stroke, which just doesn't look right on this map (especially not also applied to buildings).

How big would you suggest I make the map for printing in a book and also hide the problem of jagged lines? (The author hasn't specified how large it actually needs to be despite asking a few times so I can't give more information than that. I'd guess it's for a regular hardback, though.) I started out with a 2000x2000 canvas at 150ppi with intent to sketch the layout, but I'm used to creating for digital only so I have no idea what size to up it to now.

Thanks again for all your help and suggestions. I really appreciate it!