From the very beginning of my AI and Games video series on YouTube, I researched various AI implementations that solve specific tasks in video game design that require immediate, reactive behavior combined with long-term and strategic decision making. A popular genre in which AI can create fast, reactive and addictive gameplay are first-person shooters (FPS). But if we want to consider the need for efficient systems that take into account the balance between making short-term and long-term decisions, then an ideal subject for study is games in the real-time strategy genre (RTS).

Both RTS and FPS continue to innovate in modern video games, striving to match their design. But games in the RTS genre need to evaluate the much more extensive and long-lasting impact of quick and reactive decisions. In addition, they need to manage resources at different levels of abstraction and complexity in many hours of gaming sessions. In discussing the challenges and innovations that AI strategic game AI face, in no case should we mention one franchise - Creative Assembly's Total War .

Since the release of the first part of Total War in 2000, the franchise has steadily grown and gained strength. Each game became more and more ambitious, and the problems that the AI ​​system had to face continued to grow in scale and complexity. As a result, the company has assembled a development team that, like no one else in the history of video games, develops new trends in the creation of artificial intelligence. However, the path to the latter parts of the series, such as Total War: Attila and Warhammer , was not smooth. Therefore, in this article I will consider not only the fundamental principles of AI of the Total War series, but also how each new part of the series has improved, corrected or completely rewritten parts of these systems. In addition, I will talk about the reaction of fans of the series to these changes, the modders community that has grown in full color in the framework of Total War, and how the Creative Assembly has applied advanced scientific discoveries since the very first game of the series to create innovations.

RTS games and AI

But first, let's take a look at the Real-Time Strategy genre itself in order to understand what place Total War occupies in it. The main task in real-time strategies is usually control over territories and resources: two or more opposing sides appear on the same map in order to seize dominance, guarding resources or locations in a limited area. Often, to achieve these goals, each side builds buildings and units that either improve the collection of resources in a given area, or protect existing buildings and land, or explore and capture other areas of the map by force. In the process of solving the last task requires the destruction of enemy units. For this process, the player usually has to explore areas previously hidden by the fog of war in order to locate enemies and resources. In addition, over time, each of the parties must upgrade buildings and units, improving their characteristics and the amount of collected resources.
')

Although released in 1992 by Westwood Dune II: The Building of a Dynasty and did not become the first RTS game, it became an archetype for future games of this genre, such as the series Command and Conquer of the same Westwood, Age of Empires , Total Annihilation , Dungeon Keeper , Homeworld , as well as Blizzard's Warcraft and Starcraft franchises. The genre continued to evolve and create offshoots since the late 1990s. There appeared his similarities, for example, the genre of real-time tactics in the face of the games of Dawn of War and Star Wars: Empire at War, in which the emphasis was on tactics, rather than resource management. Meanwhile, games like UFO: Enemy Unknown (1994) explored the balance between making decisions in real time and step by step. Her approach proved popular, as proved by the successful restart of the franchise called XCOM: Enemy Unknown in 2012. One of the most distant games from the RTS was the Multiplayer Online Battle Arena genre, or MOBA, popularized by the Defense of the Ancients mod for Warcraft III and enhanced by Dota 2 and League of Legends . In them, players take on the role of ground hero units, defining large-scale strategies and implementing them during the match.


The AI ​​was used in the RTS since the very beginning of the genre: the players who controlled the movement of units simply told them where to go, but did not pave the way on their own. Therefore, the player had to interact with such systems in order to get the desired results. In addition, AI is critical for single-player game modes, in which a player fights against enemy commanders controlling their own units and production strategies. The role of AI in such games is extremely complex and is considered one of the most enormous problems encountered in scientific research. This led to the creation of many scientific assessment tools and projects, including the Starcraft AI Competition, held at the AAAI conference on artificial intelligence and interactive digital entertainment. This topic is interesting in itself, but we'll talk about it in other issues of AI and Games.

Total war

So what makes Total War so special and difficult for AI systems? A series of strategic games Creative Assembly is interesting in that it combines several modes of strategic gameplay: players step by step manage resources and policies, and also participate in large-scale real-time battles. In addition, the battle is conducted at the micro and macro levels, requiring from the AI ​​not only control of individual units, but also more abstract control over the types of troops, control of battle formations and placement of units on the battlefield in real time.

The first part of the franchise, Shogun: Total War , balances between a battle simulation that strives for realism and persuasiveness and a political strategy that gives each conflict a context and understanding of what is at stake. The action of the first Total War takes place in 1530, in the period of Sengoku feudal Japan: this time was popularized in modern culture by the works of Japanese film director Akira Kurosawa. Films such as Shadow of a Warrior , Seven Samurai, and Ran have influenced the design and development of the project. Excerpts from Run were even used in the game's videos. The player and enemy AI take on the role of daimyo - feudal rulers, who govern the provinces of Japan. They have to lead and diplomatic games, and military operations. When opponents are drawn into a conflict, players take control of the “Taisho” (by the generals) and move hundreds, if not thousands, of soldiers on the battlefield. The Sengoku period was an ideal historical epoch for the game, because politics and even the economy of that time were based on the logistics of the armies that defended the daimyo and his ambitions.


Total War differs from many traditional real-time strategies in that it abandoned the mechanics of fog of war, as well as resource management, in the combat unit. Because of the structure of the game itself, the AI's opponent manager was split into two separate AI systems:

• AI campaigns and diplomacy manages the strategy of the game in turn-based mode and is responsible for moving armies around the map, engages in diplomacy (sends murderers or negotiators to eliminate opponents or enter alliances), and also takes care of the agriculture and infrastructure of its own provinces.
• Combat AI commands combat units, strategy, and attack patterns. Like live players, he is responsible only for managing the strategy of individual groups of units, because individual units themselves are also controlled by their own AI .

Therefore, in order to deal with Total War AI, I will explore AI systems from the lowest to the highest levels: from individual unit control systems to strategic systems whose task is to conquer all feudal Japan.

AI unit behavior

Fighting in Total is conducted by units: groups of certain types of troops that can be given combat formations, from melee units to archers and cavalry. Such combat units not only have to keep building, but also move, as well as fight as a single squad. Movement is often challenging, especially when a unit is ordered to go through different types of terrain, such as mountains and forests.


Standard neural network pattern

Neural networks operate as a weighted summation function for incoming data. By changing the weights of each individual computational node, or “neuron,” we can change the result of the data transmitted to the layers on the right. [Image source]

To achieve this, Total War uses artificial neural networks to control individual units. Neural networks are a fast and efficient tool for achieving fast and reactive responses to a predetermined task. The neural network processes data in “neurons”: simple processing nodes that receive incoming data through weighted “synapses”. By changing their weights, we change the results of the data transmitted over the neural network. Typically, these weights are selected using machine learning methods, but they can be customized manually if the network is small enough. They are effective because, after learning, the processing time for decision making is incredibly short. In addition, a well-trained neural network can generalize its decision making process, that is, it can recognize similarities in a variety of individual conditions and in each case give similar answers.

Despite this, neural networks are usually not very good when they are required to simultaneously perform several tasks, especially when these tasks conflict with each other. In this case, the units have different neural networks, which they can include depending on the task to be solved: the choice of building, avoiding enemy fire or selecting a position to attack close or from afar. Each of these networks is created in advance, it is not configured and is not optimized during the game, that is, it does not learn to become better as the game progresses. Such systems of AI units work regardless of who manages the army - man or AI, because the player in any case needs such systems of neural networks to control the behavior of individual units.

Combat AI behavior

The level of combat is the AI ​​system that controls the opponents of the player. This AI makes decisions about the choice of building groups and the actions of units. He mutually moves parts of the army in order to achieve a certain tactical or micromanagement movement of units depending on the current terrain or weather.


To achieve this, Shogun uses a logical system to make certain decisions based on the current state of the world. Such decisions are taken from the "Art of War" : a collection of strategies of the Chinese military strategist Sun Tzu, dated about 500 BC. Using a logical system is well suited for text adaptation, in many cases, given conditions are used to get a certain response from the system. These rules take into account the type, condition and size of their own units and opponents, the current relief, the weather and the morale of the troops. Here are some of the examples taken from the "Art of War":

• If you surround the enemy army, leave one side open.
• In open areas do not try to block the path of the enemy.
• If you have ten times more strength than your opponent, surround him from all sides.
• If you have five times more strength, attack him.
• If you have twice the strength, divide it into parts.

The developers say that the combat AI system has implemented 220 rules in three different versions, each of which has slightly different sets of behaviors. The most important reason for this was to create as realistic a battle as possible, unlike the tactics that controlled most of the real-time strategies of that era. This is what Creative Assembly Development Director Mike Simpson says:

“In most games, simple scripted behavior scripts are used, which are triggered by contact and very simple mobs of units. They are very limited and seem to be "toy" rather than real. I mean that the tactics used to defeat them need to be learned in every new game, and sometimes in every scenario.
or level, and it has nothing to do with reality. We aimed to create a game in which real tactics from the real world will work. We chose not the easiest path, but he brought a great reward. ”

[ Battle AI , Mike Simpson, Total War blog, November 10, 2009]

AI campaign behavior

The last layer of the Shogun AI systems is the campaign and diplomacy system. She manages all decisions on the main campaign map: managing army movements, maintaining diplomacy between the parties, and building provincial infrastructure.


It uses a combination of classical decision making on state machines with genetic algorithms to create the individual tactics of each daimyo. A simple state-action system simply indicates what needs to be done under current conditions and provides an instant AI response. It does not store any memory of existing behavior and does not have a complex decision-making process. It simply responds to events occurring around. Then a genetic algorithm is used to provide some personality.

Genetic algorithms are a type of machine learning, in which we create huge numbers of possible solutions to a particular task, which learn to solve it by growing the best ones. These possible solutions are not program code, it is simply a coding of how the AI ​​usually behaves as a string of numbers, which is then passed to the AI ​​itself to set its behavior. They allow us to create sets of complex solutions to the problem: they all do not act similarly to each other and have their own slightly different characteristics. As a result, each daimyo received his own unique behavior in battle. From the players' point of view, this gives them a certain personality: some daimyo are more eager for diplomacy, others are more aggressive and expand their territory, attacking other AI rulers and players.

Medieval: Total War

All these three systems are the fundamental principles of Total War AI: they control individual units, provide convincing behavior, group troops in a logical, reactive and difficult way for players, and also create strategic opponents with their own individuality. Despite the fact that the first game was created back in 2000, the basic principles of combat and strategy, combined with the systems created in Shogun, were transferred by the Total War franchise to various new battlefields of the many parts of the game that followed. Despite all these behind-the-scenes innovations, AI still cannot be called infallible, and in every new version of the game it is improved and matures.

The first changes were made when the series was transferred from feudal Japan to the European Middle Ages. Released in 2002, Medieval: Total War sends the player even deeper back in time, at the end of the eighth century, and holds it up to the fourteenth century. Players control various parties in different historical periods: from the Viking era in Britain, when most of the country was fragmented by the Roman Empire that left it, until the late Middle Ages, when the borders of the countries of most of Europe formed.

Medieval: Total War in many ways has become an improved variation of the first game. It is built on an improved version of the engine Shogun, which allowed to hold even larger battles. There were no major differences in AI technology between Shogun and Medieval , only a lot of optimizations and fixes. AI unit systems have learned to better cope with clusters in narrow places, such as bridges. In addition, the set of combat AI systems expanded from three to nine, providing greater flexibility. All of this resulted in the creation of a game that Rock Paper Shotgun called "almost perfect." Rob Zackney wrote:

“I'm still not sure that in the Total War series it will be possible to create an even more balanced game. The AI ​​easily manages the troops on the map, and the different starting positions of each kingdom and empire allow real AI superpowers to form, which later cause the player a lot of trouble.

Medieval is a triumph of simplicity, and Total War took another ten years to get closer to him again. ”

[ The Best And Worst Total War Games , Rob Zackney, January 4, 2016]

Finally

Shogun and Medieval were the very beginning of the franchise, and it can be said without a doubt that the road to war was not smooth. With each subsequent version of the franchise, the complexity grew in scale, and the volume of the games themselves increased significantly. In the second part of the article, I’ll talk about the modder community that came about after the release of Rome: Total War in 2004, how the release of Medieval II: Total War encouraged developers to rewrite and rebuild in Empire: Total War , and how large-scale change affected the entire franchise until the release of Total War: Shogun 2 .