I'm after a particular effect for a dungeon I'm designing, almost a style or direction I may take all future dungeons I do down. Essentially, I've got the multiple layers using masks to bring out the dungeon shape; the bottom layer is the floor plans with the grid, and the top layer is typically the surrounding texture of the dungeon, the 'dead space' if you will.

The effect I'm after is having a texture expand out from the selection and then fade of; as if an opacity gradient is slowly fading the texture itself. I've used a layer mask, and brushed the opacity in myself, but when I come to the layer effects, and attempt to stroke the edges (typically for the inside wall effects) it just goes horribly wrong. I was wondering if anyone had any clues or advice for improving the effect.

I got the inspiration for trying this technique from another poster here, so essentially it's the same outside wall effect I'm trying to emulate. The selection he's used seems very accurate for the pattern gradient on the outside walls, so I know he hasn't done it by brushing it himself. I'm also aware it's done in Illustrator. I'm attempting this in Photoshop CS5, and would like to stick to that software if applicable. This is the thread with the technique; http://www.cartographersguild.com/du...lustrator.html