AI News, Steamin yhteisö :: Total War: ROME II artificial intelligence

Make Peace, Not War | The AI of Total War (Part 4)

with a campaign map of similar size and shape that succeeds the rise of the Roman empire in the previous title, only to see it crack and crumble with a total of 56 factions appearing as either ally or foe in the original version of the game.

A big part of this was that state space coverage becomes increasingly difficulty the longer the game progresses: the number of AI factions begins to grow, the number of possible future states grows at an exponential rate.

This required additional analysis of a given state using metrics from the developers itself, to determine whether a target was unreachable, a battle was unwinnable, there was a low probability of a strategy succeeding or a given path in the tree was redundant and then removed it from the decision making process.

In addition, this all required aggressive patching of the pathfinding system to pre-cache some calculations and simply reduce the number of calculations taken, only asking if the MCTS decision making process deemed it necessary.

Diplomacy is primarily interested in answering three key questions for the campaign AI on a given turn: These questions are valid both for an incoming or outgoing deal, given it wants the other faction to accept its proposal, but conversely it may need to haggle an incoming proposal to its satisfaction.  The Diplomacy system is driven by the same information and logic as the player, it can’t take any shortcuts, even when two AI factions are dealing with one another.

Each of these systems utilises four key metrics that are generated by the system given the incoming information it has received: If diplomatic value is negative, it means that everyone else really won’t like you signing this deal.

For deal evaluation, all of these values are then pumped into a weight summation function to give what is called the Deal Value: if the deal is scored as higher than 0, it means that it is worth the AI player accepting that deal.  Meanwhile in deal generation, it needs to make some smart decisions on what to offer.

The list of all possible diplomatic actions is took long to consider, as such the system will use a pre-filtered list based on the current strategic situation, factoring strategic and economic values pertinent to the current state of the game.

The actual discussion between faction leaders that takes place during negotiation is powered by a separate system that ensures that the dialogue fits the style of the people of a given faction.

In addition, many of the personality swaps are tied into the progression of faction leadership: with new personalities adopted as a faction leader dies and their heir comes to power.

The scope and complexity of Total War continues to be vast in scale, to a point that the number of minor AI systems continues grow in order to support the unit, battle and campaign AI.

While these have been critical to the series for many iterations, I’m going to take a look at the more recent innovations brought to this system alongside some fresh perspective for the franchise: as history is cast aside for fantasy and we enter the world of Warhammer.

Revolutionary Warfare | The AI of Total War (Part 3)

As the core systems of Total War have been established and redefined in the franchise - a point I have discussed in the first two parts of this series - there is always a need to strive for better.

One of the new problems this creates for players is that the army and naval logic were until that point separate, meaning the AI needed to be rewritten to consider how naval strategy could influence ground troops, such as being bombarded on the coast line.

The drive for a more deliberative system that could consider the overlap between mechanics resulted in a growing number of sub-systems responsible for individually managing the budgeting of money, conducting diplomacy, selecting tasks for attacking and defending - be they attacking enemy forces or laying siege to settlement - deciding what issues take high priority, figuring out how to navigate an army safely across the map, not to mention managing construction and taxes.

Monte Carlo Tree Search is a tree search algorithm that adopts principles of reinforcement learning: a branch of machine learning algorithms that look at a problem and find good decisions by considering all possibilities, while largely focusing on the ones it finds to be most useful.

This is really useful when you have a problem that is incredibly large and has a large number of possibilities, given we might find a good decision to make, but we can't say with any certainty it's the best decision.

We want to exploit the actions and strategies we have found to be the best, but must also continue to explore the local space of alternative decisions and see whether they could replace the current best.

This is a difficult process to resolve, given that sometimes we need to really explore a series of decisions to discover that an action that might look bad now, might actually prove to be a really good idea somewhere down the line.

This is what MCTS does best: it explores all potential options for a given decision point, isolates the best ones and then dictates which one is the best, both considering it's short and long-term ramifications.

The key component of MCTS the ability to run a playout: where the AI effectively plays the game from a given starting point, all the way to the end by making random decisions.

Now it can't actually play the game to the end - given it's happening in real-time - so MCTS uses what's called a forward-model: an abstract approximation of the game logic that allows it consider the outcome of playing action-X in state-Y, resulting in outcome-Z.

So in a context like a game, where CPU and memory resources are pretty tight, if it needs to stop evaluating the game at a moments notice, it will still give the best answer it could within that time.

MCTS is responsible for managing two critical components of the task allocation systems: the distribution of resources such that the AI can approach different tasks it wants to complete and the execution of specific tasks.

In addition, task viability also carries some filtering to stop it trying to do anything too stupid, such as removing actions that could cause diplomatic tensions, filtering actions that could impact long-term strategies and also factoring what it had done recently so it avoids contradicting itself.

Most notably, the campaign AI took quite a long time to make its decisions in the launch build: taking several minutes to conduct campaign movements that most players conduct in a minute or two, resulting in aggressive patching of the game for several weeks after launch.

But it is nonetheless a major milestone for the development of AI systems and practices in the commercial video games and has led the way for many a successor that is seeking to adopt MCTS as part of its own AI toolchain.