I've played around with bryce a while ago, and I think it might work pretty well for the effect you want to achieve.
Even the older Bryce version 4 had a simple but rather powerful terrain editor - you can imort gray scale images, edit with a brush, add filters like eroded cliffsides or smoothed out hillsides. very fun to use and very intuitive.
The down side of this is that it would be comp generated, and a lot less cool than your current hand drawn style.
Anyway Riven was one of my favourite games, and It was great looking around your site!