Quality guaranteed: What the game tester really looks like (Part 1)

The old Westwood college ad became a bit of a joke in the world of video games. Two guys, comfortably sitting on the couch, are killed in the trash controllers, enthusiastically playing on the Sony PlayStation. A girl comes in and says, “Hey, guys, have you finished testing this game? I have one more here. ”

"We just passed the third level, the graphics need a little pull up," - says one of the guys. Then, turning to his friend, he smiles, as if he had just won the lottery: "I can not believe that we play games, and we are still paid for it."
“I know,” answers the second. “And my mother used to say that my passion for video games would not lead to anything good.”
')
This is exactly how people have been imagining the lives of those who test computer games for a long time - not as a job for 5–9 hours a day, but as a dream of all teenagers. Who would not want to sit on a comfortable sofa and play games all day with a few interruptions to “pull up” the graphics in the third level?



The reality is a little different from this picture. The so-called quality control of computer games (QA), that is, their testing (hereinafter the author mixes up the process components - testing (basic level), quality control and quality assurance - approx. Translator) , is often perceived as “playing games, and you still get paid for it ", but in fact it could be better described as the process of" breaking "games. This is a low-paid, rarely rewarding and often disappointing job that affects - one way or another - the quality of modern games, but not in the way you might expect.

A professional tester does not just sit in front of the TV and, drinking some energy drink like Red Bull, goes through the fifth level of the last shooter. He (or she) spends 14 hours in a row, attacking the walls in these levels in order to check their integrity. Good video game testing is more like solving a puzzle, than getting a new record in Donkey Kong, no matter what is shown in commercials. “In order to do a good job in the QA world, a specific approach and a special attitude to life are needed,” an experienced computer game tester told me. "This goes beyond the passion for video games and certainly does not coincide with the idea that you play video games and get paid for it."

Usually, testers are underestimated, remembering about them only when something goes wrong. QA-professionals argue that this work is boring, hard and often seen as an opportunity to get into other areas of game development, rather than a more traditional career path. Often, testers work on temporary contracts or for outsourcing companies that interfere with their direct communication with game developers. And when there are especially many bugs in the game or it is generally published in a raw, almost unplayable form - as many of the recent releases - usually all testers are blamed for it. They, among other things, and those who must guarantee protection, being the last wall between the mistakes of programmers and the money of buyers. The essence of the process conveys the whole point: quality assurance. In other words, testers must ensure the quality of the product.

But are those who test the games really guilty of entering the market raw? How is it possible that testers do not find the bugs that we see in games? Why does a lot of servers lie now and then? What do these people do during the whole working day?

In an effort to explore the world of testing computer games and try to explain what this work is about and what it is like, I have been actively communicating with a huge number of people who are currently in testing or were once testers for the past few months. Many of them chose not to call themselves to protect their careers. Some said they hate testing, while others said they could not imagine themselves behind another occupation. Almost everyone agrees that only a few understand exactly what the work on quality assurance consists of.

How many games exist, so many bugs live in them. Some are relatively harmless and even become legendary, like the mysterious MissingNo in Pokemon. Others have entered the history of video games: the endless levels of Minus World in Super Mario Bros., which can be reached by passing through the wall. But the tireless members of the gaming community do not sit still: new bugs are constantly there and are worn, and also amuse the players - glitches in Legend of Zelda: Ocarina of Time, for example, was enough for a 17-minute funny video !



All these are rather friendly bugs. Most video game glitches are annoying at best and stop the game at worst. Therefore, each game and passes the quality test - extensive testing, conducted in order to verify the correctness of the toys. The QA abbreviation has come to the video game industry from the world of products - microwaves, machines, pipelines - and in many ways testing games is no different from testing products. The job of the verifier is to pinpoint and search every corner of the game and play enough of it to hell until all the glitches are removed - exactly like a factory worker who refines the last toy.

When it comes to QA, the game industry does not have any standards here: all games are different, and each company has its own approach to testing. But the tester usually spends months after the game, checking the current versions of the product being created in a variety of ways. The more bugs a tester finds, the more he is valued by the company. This, of course, is very difficult: video games are complex sets of interacting systems that need to be carefully and methodically tested for bugs. And this may include re-passing the same level with small changes (whether using a new hero, another weapon or choosing a new road) and keeping records of everything that happened to you.

Let's take Grand Theft Auto V as an example. In the vast open world created by the developers at Rockstar Games, testers had to divide and conquer. “During the tests, different people were engaged in certain missions or tasks, mini-games, etc.,” says the person who helped test the game. “Usually the work went from the general to the particular. First you go through the basic missions in order, then there are thefts, then additional missions and checking various characters, then you advance to testing the strip club and prostitutes. ”

The same tester said that sometimes they also have to spend a lot of time on tiny parts of the game. For example, when designers from Rockstar asked a group of testers to check everything that players could do with a taxi service in the game. They quickly found that if you take a taxi for a new mission, the mission starts even before the player releases the taxi driver with the world, which led to funny moments when the car circled around or tried to turn back during the in-game cut scenes.

“I think that such work on projects makes them much better due to the fact that we find such moments when something really stupid happens,” said the tester. “We found a lot of bugs: talking pigs, now and then as human beings standing on their hind legs and walking away, simple passersby, who unexpectedly soar upward. Trevor, taking off his pants, did not bother to dress them back - he ran for the rest of the game with his trousers hanging around somewhere below. Franklin's dog died when he barely touched the water ... the dog just fell into the pond and went to the bottom with a stone, as soon as it only wet its paws. ”



Finding bugs is just the first step. The second, and much more complicated, is to try to reproduce the glitches so that the company's engineers can fix them. The tester cannot simply write something like "pants fall off Trevor" and send it to a team of programmers. What can engineers do with this information? In order to find, isolate, and correct a bug, a programmer needs to know exactly how this happened, which can be a difficult puzzle, given the huge number of different factors in video games. Good testers quickly learn to memorize their every action — significant and minor — so that they can at least try to reproduce any bug they have encountered. “I like the fact that a tester’s job is like a paid puzzle solution,” says Rob Hodgson, an experienced tester with 8 years of experience. "For some people, trying to reproduce, step by step, some strange mistake that was found earlier can be exciting."

Regular tester working days can vary significantly depending on the project, role and position in the company. So, a person who got a job through an outsourcing company can spend 10 hours crashing into each wall in the latest version of Call of Duty to find out where the design can be pierced (a kind of “shock test”). A staff member who works on tests can work with a programmer, trying to figure out why the frame rate on the Android version is reduced in their mobile game. Inconsistent and, as a rule, monotonous in nature, work in the field of QA may carry some unexpected tests. For example, testers who worked on the Rock Band music game said that the sounds produced by plastic drums infuriated them to such an extent that they had to make a rule: no instruments on Tuesdays and Thursdays.

During the game, testers record reports using software like Jira to explain what happened and how it happened. Programmers, who ideally are not working at the moment on new content and are engaged exclusively in fixing bugs, analyze reports and respond, if there is such a need, sometimes with questions, problems and stinging comments.

When the game on the console is almost complete, it must be certified - a process in which the publisher (EA, for example) asks the console manufacturer (Sony, Microsoft or Nintendo) to check the game for serious bugs. During this certification process, the second wave of QA-personnel (the guys are called “conformity testers”) passes through it again, checking whether everything is in line with expectations. Each console manufacturer has its own list in which all requirements are described - from error messages to achievements, and if the game does not correspond to any element, then the publisher will have to correct it and try to get certified to again - to hell all deadlines! “Microsoft requires all games to be able to go to the Xbox 360 menu from anywhere in the game,” said one tester who worked for a major game publisher, checking them for compliance. “According to Sony’s rules, games should not be able to skip screens with studio / publisher backdrops at the beginning of the game when first viewed. Nintendo does not want foul language in their games, so all texts have filters that we check and try to break. ”

"I have not played BioShock Infinite for at least two years after release," one former tester told me recently. He worked for the 2K company and spent a lot of time testing this game, he was left disappointed with what the product turned into, which he noted was not worthy of the original version.

“The only thing that made me play the game again is the observation of the high-speed passage of BioShock. We spent a lot of nights for the fast passing game. It’s interesting to see what players do to cut levels. ”

One evening, the tester saw a video in which the world record for passing the game was broken. At the eleventh minute, he just lost his temper.



“I got mad because they use a bug to get out of the level and automatically move forward. I SHOULD BE TO FIND THIS Glitch! ”The soap tester wrote to me.

For a very long time, the video game industry has been testing as a dream job: hey, children, play games all day, and we'll pay for it! But in recent years, various “horror films” about this profession of dreams have crawled to the surface: testers began to share stories about monotonous exhausting work and bad attitude towards them from companies that perceive them as consumables in a huge developing machine.

This is also reflected in low wages. Such work does not have high requirements - usually in order to get an entry-level tester position, you do not need to have experience or a diploma. At the same time, many people want to get this job, and that is why wages are average. In 2014, the results of a study on the salaries of the average novice tester were published. It turned out that the annual salary of such an employee amounted to about 55 thousand dollars (apparently, this is a salary before taxes - approx. Translator) , but this is the salary of full-time employees, while the majority of testers are contract servicemen who work either directly with the developer, or for companies that take tests orders from multiple publishers. Many of these contractors told me that their salaries range from 10 to 15 dollars per hour - an average of 21-30 thousand dollars a year.

Testers also say that they feel disrespectful in the workplace. Many of them (especially contractors) are forbidden to communicate directly with the developers, and all communication is carried out exclusively through written reports on the identified bugs. “It was something like an unwritten rule - we could not directly contact the developers,” one tester told me. - All communication is usually carried out through QA-leads. All communication with developers was reduced to comments in the database about errors, which is not an ideal form of interaction, in which it is easy to misinterpret a comment / question from a developer about an error as an annoyance or an annoyed remark. ”

So not in every studio: “When you provide testers with benefits, opportunity for career growth, respect and lack of fear of dismissal, it attracts the right people,” says Ariel Smith, who tests MMO games in the Cryptic studio. She told me that she loved her job, but disrespect for testers really became fashionable. Several testers told me that they had to use the side entrances to enter the offices where they work, and that they were forbidden to communicate with other employees. Others say that developers often mock them to one degree or another. For example, one situation is known, when a quality engineer fixed another bug, which did not prevent him from constantly sending a message like “Unable to reproduce” to the tester. In a typical studio, testers are considered the lowest layer of the hierarchy. This is partly due to the nature of the work - the tester shows others where they screwed up. It always hurts someone's pride.

“Those who test games, think only about finding bugs, developers think only about correcting these errors,” one tester told me: “They are not a team and do not work together. It's almost like a game of tennis. Testers are generally interested in the game being buggy, otherwise they will have nothing to do. Therefore, the two sides in some sense work to the detriment of each other, which is not a healthy workflow. ”

In some gaming companies, the authorities set standards for testers for errors found, and if there are fewer bugs than specified, testers may face a reduction. This creates a strange tension when testers begin to compete for who first finds the biggest mistakes. Sometimes these employees are resourceful and find ways to work more, get more and look more valuable to the company. “There were also testers who identified bugs at such a time as to be able to work overtime. If no extra hours are scheduled for the weekend, they report a multitude of errors on Friday afternoon. In some cases, this will entail overtime hours, ”one tester told me.

Testers also have to deal with other not-so-pleasant issues of the games industry, mainly related to obligations and frequent layoffs. Large developers usually hire dozens of testers before ending a large project and say goodbye to them as soon as the game comes out. Instead of celebrating the successful completion of the project with the developers, they are forced to look for a new job.

Given all of the above, an outsider may get the impression that this is a terrible dog work, but it also has positive aspects. Many testers tell me that, despite many problems in their work, testing video games can be a satisfying and unique learning experience of its kind.

“I enjoyed testing, and I would repeat it again if I had to,” says Obed Navas, a former tester who worked on titles like BioShock and Call of Duty. “Despite the fact that a tester is not the most glamorous title, and with this kind of work you risk losing all interest in video games after hours, in the end, being able to see your name in credits is worth a lot. It’s also cool to have some project-related things that you cannot get anywhere, and to questions from acquaintances about where I took them, with pride to answer "I worked on this game."

The second part is here .

PS Do you work as a tester? Do you agree with the opinion of the author of the original article? Please tell us about your work, its pros and cons, as you see it.