It looks pretty darn good. The only comment I have, and I know this would be hard to do, is that for a an old book just laying open, the lines of the map and the text are awfully straight. It looks like if I hold a ruler up to the computer screen, the text and other lines will line right up. This makes it look not so much like it's following the curve of the page.
I'm not sure how I'd do it, myself (I think there are some hints on it in RobA's GIMP tutorial on how to do distortion, if you're using GIMP) but it looks like they could use some kind of distortion.
You mention that it doesn't look right if you flatten it. Are you flattening in the right order? If you flatten your layers in the wrong order, you lose some effects, but in the right order and everything is preserved. IIRC, you have to flatten from top to bottom.