Google has revealed some
details about how the Chrome browser is tested for vulnerabilities. To do this, they use the fuzz testing method, that is, they use a testing technique in which invalid, unintended or random data is input to the program.
The idea is to test as many options as possible. Naturally, this requires serious resources. To test Chrome, a whole cluster of ClusterFuzz servers has been created, consisting of several hundred virtual machines.
About 6,000 Chrome instances run in the cluster simultaneously. The cluster automatically takes the latest build of Chrome
LKGR (Last Known Good Revision) and runs it after about 50 million test cases per day.
Google does not spare resources on ClusterFuzz: since the launch of the system at the end of 2011, its capacity has increased fourfold and is planning to increase fourfold in the coming weeks.
')
The system automated not only the generation of tests and the registration of failures, but also a number of higher-level tasks:
- Generate a smooth flow of test cases, distribute them across thousands of Chrome instances, process the results.
- Failure analysis: only faults are important, which from the point of view of information security can lead to exploits, so here the Address Sanitizer memory error detector is used to process the source code, get a special version of the binary and issue detailed reports on cases suitable for exploits.
- Test minimization: tests for fuzzing are often very cumbersome files, usually several hundred kilobytes each, so that after the generator these files are broken up into smaller, meaningful fragments.
- Identifying regressions: looking for changes in the source code that caused the crash.
- Patch Verification: Check for previously detected failures on a new LKGR assembly.
Since the launch of the ClusterFuzz cluster at the end of last year, he discovered 95 unique vulnerabilities in Chrome test builds. Of these, 44 were identified and closed even before this code reached a stable release. After upgrading the system, Google hopes that before entering a stable release, it will correct even more errors both in itself and in open source projects on which Chrome is based. This, including,
WebKit and
FFmpeg .