Try to think about what information you are trying to present. There will be shapes (features) and information about them (attributes) which you present in a particular way(symbols). You want a clear distinction between symbols that mean something, and decorative elements. Try to avoid clouds and textures that look like they are trying to be symbolic. If you just have "water" without any attributes like depth, it's best to present it as flat and even as possible, with maybe a simple coastal effect to help emphasize the coastline.

It's a good idea to get the features of a map complete before you even start styling it. You might find things go better if you have everything placed before you worry about water textures.

If you are trying to mimic a had drawn "in character" fantasy map. Consider the tools, techniques, data and goals of your notional cartographer. Why does the water have the texture it does? Is it the underlying texture of the paper/parchment/whatever? Is it an unevenness in the medium used like a watercolour wash? Is it aging, wear, or other "distressing"? Or is it an intentional part of the symbol like stippling or a wave pattern?

Remember that a map is a functional thing, not just a decorative wallpaper. If it's hard to read, it's a bad map regardless of how pretty it is. (It might still be a good map inspired image though).