You can try adjusting the highlight and shadow opacities, colors and modes, but you risk reducing the contrast in the good part while fixing the bad.
The trouble is that you're simulating throwing light on an object. Stuff that's in shadow is going to naturally be darker than stuff that's directly lit. You could try setting the light source right in the center of the circle, which will simulate light coming straight down from above. That will reduce the sense of the texturing, though, and you'll likely lose much of the effect.
You could segregate your walls into layers with different light sources, setting a light source that's angled roughly parallel to each wall (your screenshot shows that that's what you have here. If you draw a line from the light source through the center of the circle, you'll note that the angle is approximately parallel to the walls in the well-lit canyon. You'll have discontinuities where the two bevels but up against each other, but you might be able to disguise it with some soft-edged masks. This approach will probably look a bit wonky, since you'll have light coming in from more than one place. It could work if the map is underground, but it will definitely look strange in day-lit terrain.
Or you could take a hybrid of the first and third approach, leaving the light angle alone, but changing the shadow and highlight values on walls that are too dark or too light.