If I remember right glass windows became more common amongst the non noble classes in England mainly during the 1600s. When nobles built or rebuilt their manor houses (usually in brick) during the tudor period they would often include lots of windows - it being a prestige thing. Of course when the window tax came in at the end of the 17th century a lot of windows were then bricked up on smaller estates.

Of course defence is going to play a big part - you probably aren't going to put lots of big glass bay windows on a building designed to withstand a siege! (But you might alter it in such a way later on.) Climate also makes a big difference as Falconius mentioned.

There are other materials that can be used for windows though such as alabaster (and other translucent stone) and animal skin which also has translucent qualities when stretched. Both were quite popular on roman buildings. If I recall correctly a lot of the more downmarket insulae (apartment buildings) just used translucent stone windows especially on the upper floors.