Unity 3D Test Automation Team
Hello everyone, my name is Elvis Alistar, I have been working with Unity for over 2 years. I am the head of the Testing Engineers (IT) team, which is part of the QA Unity department. As the name implies, we are a team of software developers who love to test. In this post I would like to tell why it is so important for Unity to have a team of experienced developers focused on test automation.
Unity is a fast-growing company with over 450 employees from 50 countries operating in 27 locations around the world. The large distribution of our company means that we need to work with developers from different time zones, different cultures. We also have over 2.5 million registered developers, and we often interact with them through our forums, feedback website, alpha and beta groups and processing some of the error reports we receive from them. Working closely with our development teams means that good communication skills become key.
')
Unity supports the 12 largest platforms and several smaller ones. Users can also use three different programming languages to write game code. Add to this the presence of runtime tests, graphics tests, integration tests, module tests, UI tests, performance tests, etc. - and the phenomenal number of different combinations that our test automation infrastructure should support and manage.
Take our runtime tests, for example. Now we have more than 1,300 unique tests of the Runtime API. Over 13,000 tests are performed in a single automated build of the test set. At the same time, we work in more than 100 areas of development, which means that there are about 500,000 control points performed on our build farm every day. IT should write, maintain, optimize and improve it daily.
Just two years ago, we had only a few test platforms that were very poorly documented, we had very few tests, and no one supported or improved the platforms or test code. An IT team was formed to take on the support of these platforms, put in order, document and optimize all our tests. We also made sure that all developers know how to use these platforms and that they can easily create new tests using them.
The code base in Unity is growing very fast, and we need to be sure that the product is very stable with every new release we release. That is why we pay great attention to the automation of Unity testing. Just to see the perspective, we now have about 2.2 million lines of code, of which approximately 400,000 are test code, divided into more than 7,000 files. This ensures that we can maintain quality at a high level, avoid new regressions, and make sure that known errors that have been fixed will never appear in our product again.
Now what I call a task!
We are a team of 8 developers working on tools, platforms and automation features, with more emphasis on the latter. Two of us are in Copenhagen (Denmark), one in Silicon Valley (USA) and five in Odessa (Ukraine). Our job is to find ways to improve the overall workflow for all developers on Unity, by improving the automation on the build farm, creating regression sets to prevent pop-up errors, and promoting the use of the platform by testers and developers.
Unity is a very complex product, and in order to work more effectively we have divided it into different areas: Scripting, Lightmapping, 2D, Animation, etc. Each IT is responsible for writing, maintaining and reviewing tests in one or more areas. New IT s usually start from one area and work on it until they become proficient in using our processes, platforms and strategies. After they gain enough knowledge and trust they will gain more areas. All ITs are ultimately working on improving our existing platforms or adding new ones. We have 7 different types of automation platforms in Unity, and IT should be able to work with all of them and help those who need to work with them.
Our work requires close cooperation with other teams in the QA structure and teams in our R & D department. We must communicate with software developers and testers (TPO) to make sure that we know who is working on this feature, which parts have been developed and tested, and what should be automated. We all participate in basic QA tests in each release, even if they do not include test automation: exploratory testing, full testing and pre-release testing (see the Testing Guide in the article “Testing Unity 4.0” in our blog).
Automation work is rarely tied to a specific release of Unity, and this gives us more freedom and flexibility to focus on what we want to improve. It also allows us to visit our various offices to help developers or to work with local testers. We attend large testing conferences and speak at them when there is such an opportunity. We participate in our now famous HackWeeks, where we can experiment with any ideas and projects that we have, no matter how strange they may seem.
As IT, we are required to have extensive knowledge in programming, object-oriented design and test automation (unit testing, TDD, high-level tests, etc.). We need to disseminate information about testing platforms and methods, maintain feedback in code analysis and be able to work well with developers. We value clean code and best practices development.
In Unity, we also need to be able to work efficiently with many of the tools we use inside, such as FogBugz, QMetry, TeamCity, RhodeCode, Sikuli, NUnit, Moq, Unittest ++, Atomiq, etc.
Like this! Being IT in Unity is both great and beneficial, and the variety of tasks that we solve is astounding.
There is a task! Are you ready for her?
Then get us a job!
Note of the translator: I will be glad to all the comments and advice on improving the quality of translations.
Task
Unity is a fast-growing company with over 450 employees from 50 countries operating in 27 locations around the world. The large distribution of our company means that we need to work with developers from different time zones, different cultures. We also have over 2.5 million registered developers, and we often interact with them through our forums, feedback website, alpha and beta groups and processing some of the error reports we receive from them. Working closely with our development teams means that good communication skills become key.
')
Unity supports the 12 largest platforms and several smaller ones. Users can also use three different programming languages to write game code. Add to this the presence of runtime tests, graphics tests, integration tests, module tests, UI tests, performance tests, etc. - and the phenomenal number of different combinations that our test automation infrastructure should support and manage.
Take our runtime tests, for example. Now we have more than 1,300 unique tests of the Runtime API. Over 13,000 tests are performed in a single automated build of the test set. At the same time, we work in more than 100 areas of development, which means that there are about 500,000 control points performed on our build farm every day. IT should write, maintain, optimize and improve it daily.
Just two years ago, we had only a few test platforms that were very poorly documented, we had very few tests, and no one supported or improved the platforms or test code. An IT team was formed to take on the support of these platforms, put in order, document and optimize all our tests. We also made sure that all developers know how to use these platforms and that they can easily create new tests using them.
The code base in Unity is growing very fast, and we need to be sure that the product is very stable with every new release we release. That is why we pay great attention to the automation of Unity testing. Just to see the perspective, we now have about 2.2 million lines of code, of which approximately 400,000 are test code, divided into more than 7,000 files. This ensures that we can maintain quality at a high level, avoid new regressions, and make sure that known errors that have been fixed will never appear in our product again.
Now what I call a task!
Team organization
We are a team of 8 developers working on tools, platforms and automation features, with more emphasis on the latter. Two of us are in Copenhagen (Denmark), one in Silicon Valley (USA) and five in Odessa (Ukraine). Our job is to find ways to improve the overall workflow for all developers on Unity, by improving the automation on the build farm, creating regression sets to prevent pop-up errors, and promoting the use of the platform by testers and developers.
Unity is a very complex product, and in order to work more effectively we have divided it into different areas: Scripting, Lightmapping, 2D, Animation, etc. Each IT is responsible for writing, maintaining and reviewing tests in one or more areas. New IT s usually start from one area and work on it until they become proficient in using our processes, platforms and strategies. After they gain enough knowledge and trust they will gain more areas. All ITs are ultimately working on improving our existing platforms or adding new ones. We have 7 different types of automation platforms in Unity, and IT should be able to work with all of them and help those who need to work with them.
Our work requires close cooperation with other teams in the QA structure and teams in our R & D department. We must communicate with software developers and testers (TPO) to make sure that we know who is working on this feature, which parts have been developed and tested, and what should be automated. We all participate in basic QA tests in each release, even if they do not include test automation: exploratory testing, full testing and pre-release testing (see the Testing Guide in the article “Testing Unity 4.0” in our blog).
Automation work is rarely tied to a specific release of Unity, and this gives us more freedom and flexibility to focus on what we want to improve. It also allows us to visit our various offices to help developers or to work with local testers. We attend large testing conferences and speak at them when there is such an opportunity. We participate in our now famous HackWeeks, where we can experiment with any ideas and projects that we have, no matter how strange they may seem.
Skills and installations
As IT, we are required to have extensive knowledge in programming, object-oriented design and test automation (unit testing, TDD, high-level tests, etc.). We need to disseminate information about testing platforms and methods, maintain feedback in code analysis and be able to work well with developers. We value clean code and best practices development.
In Unity, we also need to be able to work efficiently with many of the tools we use inside, such as FogBugz, QMetry, TeamCity, RhodeCode, Sikuli, NUnit, Moq, Unittest ++, Atomiq, etc.
Like this! Being IT in Unity is both great and beneficial, and the variety of tasks that we solve is astounding.
There is a task! Are you ready for her?
Then get us a job!
Note of the translator: I will be glad to all the comments and advice on improving the quality of translations.
Source: https://habr.com/ru/post/230907/
All Articles