If I understand you correctly, you essentially want to erase the portion of the drop shadow that is north and west of each wall, leaving just a shadow to the south and east of each wall.
Duplicate the walls layer, shift the entire layer one pixel north and west, and change the black walls to white. Place this below your original walls layer and above the drop shadow and white background. This will erase the first pixel width of shadow that you want to remove. Repeat this process to erase further and further out.
I think that should do what you want.