I'm very interested in what you are planning to use for developing this system. I would love to see you keep a development journal for everyone to read and maybe share ideas. Have you picked a language or any packages to work with yet?


I'm thinking a system that takes a rough, low res height map + tile data and then generated refined grids only in areas your AI actually uses may be a good path. This could allow you to do fairly decent sized regions without needing the entire high-res data set to work with, so you save yourself some hard drive space. (Especially useful if you want to save milestones of your simulation so you can go back and nudge things later.)

Also consider how often you actually update a given AI entity. If an AI has their home, their fields, production levels, and trade agreements, then you can basically freeze the majority of their AI and let the program focus on the trade alone. Pathing between their usual routes doesn't need to be updated, just stored for density factor. (If too many other AI attempt to use the existing paths, then it may trigger an upgrade to the road, or force other AIs to move away/find new roads.) Focus on the 'settler' AI, and keep a list of established AI and only go back to them when they reach given thresholds of wealth/resources.

If you get too detailed too quickly, then you run into the "Dwarf Fortress" problem, where the system can bog down with only a few hundred agents running around. Abstractions are key.