I have been working in the gaming industry for over 8 years. In Wargaming I solve a very wide range of issues: from analyzing patches to solving various internal problems. So it turned out that the problem of comfort and performance came to my attention. After talking with the developers of World of Tanks, testers and researchers of the players' behavior, I had a vision of the issue, which I wrote in the form of this article. I am not an expert in computer graphics or a graphics engine developer, I do not do UX tests on people. But I hope the collected information and my unbiased opinion on the question will push you to progressive thoughts and stimulate you to creatively find a solution to the performance problem in the game.

What is performance?

Typically, under the performance understand the average level of FPS in the game for a certain period of time. This suggests that high FPS means high performance and vice versa.

This simplified picture hides some important details. For example, the fact that for different game dynamics the requirements for the FPS level may differ. The perception of a first-person shooter is much more sensitive to performance than the RTS. Even the effectiveness of performing certain actions in a single game (moving, shooting, staying in a hangar) can put forward different requirements for the FPS level.
')

Graph of the effect of FPS on the effectiveness of actions in the game. The solid line shows movement, the dotted line is shooting. The vertical axis represents normalized efficiency. It can be seen that the effectiveness of shooting depends more on FPS than moving. Source: shooter games. Kajal Claypool and Mark Claypool

In cases where the average FPS level remains unchanged, short-term failures (15 seconds or more), delays (lags) and twitching can spoil the impression of the game and seriously reduce the effectiveness of the player’s actions. But this conclusion is valid not for all game situations; Moreover, it does not apply to all games.

Thus, performance is a multifaceted phenomenon, and its perception strongly depends on the rules of the game and technical conditions. The same FPS level in different cases can be perceived as low or high. A high average FPS with failures can be perceived worse than a low but stable one.

Performance Factors

PC configuration. Everything is obvious here: a powerful “iron” means better performance.

Client software, including OS. The main source of performance problems in games are antiviruses, torrent clients and other background programs and services that access the disk during the game. The important point here is the lack of awareness by the user of the source of the problem - the game is always to blame.

Network problems. This category includes all causes of poor network functioning between the server and the user's device. Poor network equipment, low Wi-Fi signal or a large number of wireless clients, problems on the provider side, non-optimal network route of traffic delivery. In general, all that creates problems is packet delivery.

The speed of the application. Different game engines consume a different amount of resources, depending on the quality of the architecture and the supported effects.

In addition to the engine, some load creates interface elements. As we will see later in the example of mods, parts of the interface can be complex software components that consume a significant portion of computer resources.

Graphics settings. More than half of our users customize graphics manually. Not always, these settings are optimal.

Automatic detection of a preset is also not perfect. The range of configurations of client machines is within several generations, and the number of presets is limited by common sense. High poly models require high resolution textures, beautiful effects and dynamic shadows. It is illogical to disconnect one and leave the other. Devices with very different performance can fall into one preset.

Fashion The results of recent tests conducted by our QA department objectively show that mods have a negative effect on performance. Third-party modifications of the client not only reduce the average fps, but also create sharp failures in those cases where the net client works stably.



Schedule fps during the fight. The blue line is a client without mods, the red line is with mods.
It can be seen that the mods are capable of causing serious performance failures in those places where they are not on the “clean” client.

The graphics engine is only one of the factors (not counting the settings) that affect performance, which is under the direct control of the developers. Therefore, it is impossible to draw unambiguous conclusions about the performance of the game in real conditions on the basis of engine tests in ideal conditions.

Performance Measurement Methods

Popular testing methodology:

1. Medium FPS. A large number of real players' replays are collected in the most popular game scenarios. Then the data is averaged. Used to evaluate overall performance relative to other client versions.
2. Synthetic tests (rotating camera at a fixed point). Used to search for problems in relation to the location in the game world.
3. Min on a two-second and five-second window. Used to detect less noticeable performance failures (when the average FPS has not changed) when modifying a render.

Alternative method (used by a major hardware manufacturer):

1. % of the time when the FPS is below 30.
2. % of the time when the FPS is below 20.
3. average FPS in typical game situations.

The first technique aims to identify large and medium-sized bugs that reduce performance. An alternative technique is aimed at comparing the performance of various games on a specific hardware. None of the methods allows us to assess how comfortable the user is to play, since it does not take into account a) the game context (the player aims or moves); b) the needs and characteristics of the player (skill level).

Comfort games

Thus, reasoning about performance inevitably leads us to the need to introduce such a thing as “game comfort”. In this case, a comfortable game means a level of performance in which the user does not feel its lack: the picture is updated with sufficient frequency, there is no suspension of the video stream and delays between the user's commands and the reaction of the game world.

Obviously, with a stable performance level of 60 FPS, the absence of friezes and delays, the comfort of the game will be maximum. At 10 FPS, it is extremely difficult to play almost any modern game. All other intermediate cases can not be interpreted unambiguously.

Studies conducted by specialists of our company show that for the entire set of users the connection between performance metrics and comfort is rather weak.


Chart of the relationship between perception of comfort and the average FPS. With an increase in productivity of 3 times,% of players who consider the game comfortable, increases by only one third.


The distribution of the average FPS for players with different levels of comfort differ slightly. It can be seen that the distribution areas almost completely coincide.

The results of the study suggest that:

1. Experienced players have better performance indicators, while the level of comfort they declare is no different from less experienced players.
2. The concepts of “comfort in the game” and “technical problems” are related to each other, but the absence of problems does not guarantee a high level of comfort, and vice versa.
3. If you investigate not all players, but separate groups, then there is a chance to find a stronger dependence. The perception of comfort at the same level of performance will be different for different groups of players and a clear relationship can be identified.

findings

1. Client developers measure performance, and the product owner (Publisher) should be interested in how comfortable it is to play to users.
2. The comfort of the game depends on the performance in a complex way. In addition to performance, other factors greatly influence the comfort rating.
3. Current methods of performance evaluation help developers find problems, but do not allow the product owner to assess changes in the sense of comfort of the game to the entire audience.
4. To assess the perception of comfort by the players, a special assessment technique is needed.

Links to research

The influence of FPS, camera perspectives and permissions in games on human perception of performance and efficiency in performing game actions. These studies have been officially published, that is, they have passed a peer review and are accepted by the scientific community and are also quoted by them:

1. Shooter games 2007
2. The First Person Shooter Games (Best paper award)
3. 2009: Perspectives, Frame Rates and Resolutions: It's all in the Game, Mark Claypool, Kajal Claypool
4. Is 60 FPS Better than 30? The Moving Target Selection