The cities are actually carved into the mountains; no additional construction material was used, just the sort of stuff you use in mines to dig a tunnel (except here they dug buildings and houses and streets). That being the case, I guess the cities would "occupy" part of the slope of a mountain, beginning at ground level and ending several dozen metres (or a few hundred metres in the case of the larger cities) above that. So no, no cliff.

I found a few real-life examples of similar things: Petra and Huápoca (only imagine they're on mountainsides and not on cliffs).

Thanks for asking.