I've had to do this a few times and I agree that it isn't difficult exactly. But I've never been 100% happy with the results either. The big thing for me was that the various pieces each had an extremely small amount of rotation. Too little to adjust out digitally but enough to affect how the pieces lined up.

There was also a problem with stretching. I once scanned in the original greyhawk map. It was two 22" x 34" posters, each folded in half three times which made each section 8.5" x 11". That was a perfect size for scanning but the folds caused subtle stretching that distorted the image (again, only very slightly).

What I found that helped reduce both problems was to cut up the original before scanning. I didn't do this with the greyhawk map but I did with another project. This reduced the distortion effect caused by folding and gave me common edges between the pieces to use to align them up with each other. By setting the same cut edge against the scanner edge, the rotation was as close to identical between the pieces as I could get.

If you don't want to cut up the original (understandable!) I think the key is to match the rotation as closely as you can. Also, it may help to lay the original on a flat surface and press it flat with heavy books for a day prior to scanning. That will help work out any deformations in the paper that might have occurred during the drawing process or from rolling it up, etc.

hth