View Full Version : Untitled layer mask test

09-16-2009, 11:51 PM
Actually for a test, this didn't come out too bad. The beaches were.. interesting to do. The brush I was using was doing some unexpected things, but I think that was because of the layer style combined with the soft brush combined with the mask. Perhaps it's normal for it to act strange, i don't know - this is my first attempt at working with layer masks to any real extent :)

So, a combination of paying attention to the Masks and Such with Craig John (i forget the username of who made and/or posted it, argh), a few layer styles (bevel options) inspired by pasis, whoever it was that wrote that one tutorial that included the Jiggle filter (wow, it's my new friend) and me just monkeying around, this is what ends up.

I know I could probably get the layer masks to work more efficiently in a few areas, but it was really starting to frustrate. I found a way to keep almost everything working the way I wanted to, but there was a spot where I was trying to get an outer glow and an inner glow to show up at the same time and the masks wouldn't let me. I also gave up on using the textures as pattern overlays, because they were messing up th layer styles I was trying to work with. So until I can sort out a few issues, this will do.

I took a screenshot of my layer palettes just to show exactly why i'm semi-proud of this map :) On the left are the layers as is, the right shows the expanded smart object (land layers). Yes, each layer is 100% texture filled, all terrain work is done through the masks alone. The mask on the smart layer is where the rivers came in (yay, i figured that one out, lol). This is also my first attempt at working with smart objects. Nifty keen! Though I wish there was a way to "un-smart" the layers and put them all back as separate objects/layers - there likely is and I haven't found it yet.

Ok! that was fun!

09-17-2009, 12:51 AM
So just where do you get all your textures? It seems every map I see from you has amazing texture-work that takes it above and beyond the level of a 'normal 'map. Bravo!

And I must agree: Smart Layers (and their cousins, Smart Filters, if you have CS3+) are probably two of Photoshop's most genius moves, yet. I think I've basically decided that they're almost as big as layer-based editing in the first place, though I haven't wholly decided on that, yet. ;)


09-17-2009, 01:47 AM
Hi - I'm glad you found the tutorial useful. :)
(and it's 'craggles')

but there was a spot where I was trying to get an outer glow and an inner glow to show up at the same time and the masks wouldn't let me.

This depends on how you've set up the layers. On a layer that has a mask, you can create an inner and outer glow without any problems.

The issues start if the layer in question is within a layer group that has a mask which may hide the inner or outer glow (depending on whether the layer group (the folder in the layers panel) mask is revealing or hiding the land mass) or if the layer (with the inner and outer glow) is being clipped by a layer below which also clips the layer effect.

If you like, I can have a look at the photoshop file and find the problem for you.
But you'll need to drop the resolution of the file so it's small enough to email/upload quickly. Once I find the problem layer, you can replicate the solution on your original higher res file. The layer image you've attached is handy to see but I'll also need to look at what effect is applied to each of the layers.

09-17-2009, 04:16 AM
Really, the issue came up because i was primarily trying to use a single mask for the entire map - going for the ultimate ease in editing the landmass shape. I could either get the inner glow to show, or outer glow, but not both. that was because either I was masking the inner landform, or outer ocean. I'll take a look at chipping the file down to a manageable size and upload it in a bit (RL timing right now, heh) so you can take a peek, thanks for the willingness to help a grasshopper :) (I'll see if i can replicate the effect I was after in the first place, it was a really nice ocean gradient)

alta: i'm starting to agree with you now that i've actually *really* used smart objects :) and the textures are courtesy of Genetica. I'm starting to use it a bit less now that I have a fair number of textures generated and can recycle, but I still find myself creating at least one or two custom textures for each map. It's a lot quicker and easier than sifting through my previous archives from my 3d rendering days (i had some good stuff, but most of it was for objects or wood surfaces, not groundwork or anything suitable for really zoomed out effects) and google-fu (though I still do that, old habits, lol)

09-17-2009, 05:33 AM
That looks really great. :) So good that you've made it with such a minimum of editing. The beaches look superb.

The jiggle filter you mentioned is championed by Ravells btw, so you have him to thank for that.

09-17-2009, 09:38 AM
I don't know what you were trying to accomplish, but what you did accomplish looks great!

09-17-2009, 12:51 PM
karro: thanks, looks like even my failures look cool these days :)

craggles: i figured out a better way to explain my problem.
I'm trying to use a single mask for the entire landmass (minus the mountains and forests). This involves 3 ocean layers, and originally I was looking to add 3 more land layers on top of the basic land shape for terrain variations along different parts of the shoreline. The problem I ran into was that I ended up needing to put the layer effects on 2 different layers. I think the problem I ran into was that I was using the pattern fill for the textures, and that was hiding one of the glow effects (i was trying a subtle shore outline for inner, and ocean gradient for outer). The goal i was trying to achieve was to have each layer completely solid black filled, and *everything* else being done through layer effects and mask. So due to the pattern fill (which needed to be at 100% opacity) I could not show both glows on the same layer without one getting masked out be another layer or covered by a pattern fill.

If you follow what i mean.

I compromised by switching to each layer having it's own solid pattern instead of black colour, with no pattern fill layer style, but I forgot to go back and try to add the layer effects I *originally* wanted to use now that the pattern style wasn't covering anymore.

I'll give that a try and if that still doesn't work, I'll upload that psd and see what can be done.

(and btw, i just wanted to say, that tutorial rocked. I really do well with your teaching style, there's a lot of subtle things you add in there when talking that act as "memory anchors" for lack of a better term. Wonderful job there)

09-17-2009, 01:21 PM
(and btw, i just wanted to say, that tutorial rocked. I really do well with your teaching style, there's a lot of subtle things you add in there when talking that act as "memory anchors" for lack of a better term. Wonderful job there)

Thanks - I really appreciate the kind words! :)
(It encourages me to do more despite the project being officially on hold thanks to my health).

09-17-2009, 01:30 PM
Ok, I found out why I was having the issues. It's the inner glow I need to get the ocean effect i was after. It has the option to start from Center. So I can't use the mask as the shape of the land, it has to be the shape of the ocean in order to get that effect. But that conflicts with the other layers needing the land shape to be the mask.

When I had the pattern fill layer effect, that was cancelling out one or the other glow when I inverted the mask back and forth during testing stages. I'm going to give one more experiment.

Encouragement tends to work both ways, so if i'm helping you out as you help me, then everyone wins :P

09-17-2009, 03:06 PM
That looks really great! How do make the sand look so "sandy"?

09-17-2009, 03:13 PM
a darn fine sandy beach texture, and layer effects out into the water :) if people keep asking about them, i might pack up my textures and offer them out as a zip download or something.

09-17-2009, 03:34 PM
Hahaha.. I thought as much. :) I'll have to go hunting on CG textures for som sand next time.

09-17-2009, 05:47 PM
Andartar: Here, I'll make that search real easy :) may these be as useful as they are for me!

craggles: I chopped the file down to 300x500 and it's still almost 90 megs. I think it's all the texture patterns loaded in with it. I'll give a try at recreating what I was doing with simple colour fills to kep the filesize sane.

09-17-2009, 05:50 PM
Very nice textures and a beautiful map, Coyote. Can you crosspost these textures into the mapping elements subforum (http://www.cartographersguild.com/showthread.php?t=724&page=6) please? They will more easily 'findable' there.

Glad to see you've found the jiggle filter useful. It's a gem.



09-17-2009, 06:04 PM
Actually I'm going to bunch up the stuff I currently have into texture packs and upload them as zips, so no worries there :)

As to Jiggle, I've actually had the eyecandy filters for.. almost 7 years i think. I upgraded from 5.5 to 7 and some of the filters I had at the time wouldn't move across to 7, so I gave up on all of them thinking it was a compatibility issue (i still have my 5.5 installed in case I deeded anything, but it dropped off my radar before long). the other day I was going through my archived backups and noticed my plugin backups and figured heck why not. Much to my surprise, eyecandy and xenofex installed no problem, so I figured I'd give your tut a whirl. Glad I did :)

09-17-2009, 06:13 PM
Try using the jiggle in conjunction with Pasis's mountains. Makes fantastic foothills.

09-17-2009, 06:31 PM
That's actually more or less what I did with this map, actually. There's 5 mountain layers, each with an increasing level of bevel effect (a la pasis) that i'd set up first. I selected the area within the mask, and decreased the size for each layer by 50 with a 50px feather. Then I found a jiggle setting that made me happy (I laugh every time I reread that) and just went through and applied it to each layer going down. Base layer stayed as is, and beach layer goes under. I then selected the base layer and removed the selection area from layer 3, and that gave me the area for my forest overlay. It was actually rather easy to do once I got the layer style headaches out of the way.

09-17-2009, 06:34 PM
Then I found a jiggle setting that made me happy (I laugh every time I reread that) Tell me about it. I'd be singing 'Why don't you jiggle it, just a little bit' while I was using that filter.

09-21-2009, 06:48 AM
Ok, this is a test at a larger scale - I think maybe I'll call it more than a test and do a bit more tweaking then call it finished.

And I've definitely become a lot more comfortable with using masks for terrain. I didn't even refer back to the tutorials or working notes this time!

AslanC and I are working on a redesign for his world, and I suspect we've got a finalized version at last. I'm planning on using the masking to do the entire map (8000x5000 if i recall) and do the whole darn thing in one go so it's more consistent this time. It wasn't bad before, but a few of the areas looked far too much like 2 straight edged maps pasted together (which in a way, was exactly what they were).

I'm also debating the merits of the entire layer being filled with pattern before masking, or just the continents themselves being filled in (maybe with a bit of overlap for tweaking). Considering it's the same seamless tiled pattern, would the full layer be more of a memory load?

So, we now present Atlantis (more or less)...

Anyhow, here's the Atlantis analog!

09-21-2009, 07:06 AM
It's looking very sexy now!

09-21-2009, 07:16 AM
I still haven't figured out how to get the ocean to work with me the way I want it to. I'm using an inverted copy of the land layer mask, but that has the advantage of keeping layer effects consistent when I draw the rivers onto it (I have the ocean placed above the land layers). I left off the smart object step this time, it seems to work fine without that for what I'm doing here, and things are still incredibly easy to modify, so I'm not worried anymore.

[edit] I take that back.. i have it working EXACTLY the way I want to to. Putting the ocean layer above everything else was the key. I then removed the mask on the base land layer, and now the land is editable just by painting on the mask from the ocean layer. if I want to cut into the land, it takes a bit more effort, but if the land is all placed as a smart object, it works just fine. if i need to edit mountain layers, though, I need to go into the smart object to do it is all.

Oh and incidentally, I found a way to un-group a smart object (more or less). When you edit the object and it loads itself as a new file, you just duplicate the layers, and when the requester pops up, tell it you want them to copy to the original file.

{one more edit} I have my answer regarding full layer vs only what you need. I have the original file, full layers of texture - 206 megs. I have the other one chopped down by selecting the masked area and expanding by about 50 pixes for safety margin in case of edits, and it weighs in at 101 megs. I imagine it would have a similar impact on the available ram when opening the file to work on.

09-21-2009, 09:05 AM
Looks good.

I prefer your other test map though overall as I thought the edges of the land on that one were superior to this one.
If you look at the edge where the land meets the sea it all seems to have a slight step up to the land, no beaches. To me it looks like the island originally was larger but you've nibbled away at it with an eraser set to pencil, leaving a hard edge behind.

The mountains, forests and hills look great.

09-21-2009, 09:18 AM
Funny you mention that, that's pretty much what I did :) The orginal landmass had a lot of straight edges that needed to be fixed, so i went over them quickly. Now that I'm comfortable with the style, I'll be fixing a few things like the coastline, but really, that's almost all that needs to be done now.

The beaches on the other one were very nice, I agree, but on a continent scale, you're not going to see beaches like that unless they're part of a desert. This map is a small part of a much larger area, and definitely not a desert area. that having been said, there will be some beach type areas once the detail work is done.

Good eye though :P