Yes, Roman roads tended to be straight, when they could be, except when close to extreme geography - mountains, rivers, canyons, etc. Such could be said of some meso-American road systems as well. By and large most societies did not build straight roads until late in the modern era.
However, do all fantasy world, have a "Roman Empire" that built straight roads in the past - of course the answer is, it depends. Many fantasy worlds do not have an historical advanced technology culture that has fallen into ruin, yet still have some straight roads as left-over architecture.
Comparing to Earth history, you have a point, and only limited, since not all fantasy based realms are based on Earth history, near Earth history... you can't depend upon that as a "trueism".
Interesting thoughts, though.