Smarterer - crowdsourcing professional testing

Testing is a demanded personnel assessment tool. For the selection of IT-specialists, professional testing is most often used. Many companies conduct it on their own or assign it to agencies that have a set of type tests for various roles and specializations, such as Brainbench . Recently, this niche has a new interesting project - Smarterer.com . This startup last June received a $ 1.25 million investment from True Ventures, Google Ventures and several business angels. The service officially started in October and is growing steadily .

Smarterer focuses on simplicity and speed of testing. It allows you to evaluate knowledge and skills in just a minute. At the same time, it cannot be called primitive. To determine the level of subjects, the Glyco rating is used - an improved version of the Elo rating widely used by chess players. The accuracy of the assessment increases with each answer to the test question, the system continuously adapts and improves.

The tests themselves are compiled using crowdsourcing. Anyone can create a new test or add a question to an existing one. New questions first appear in the sandbox and do not affect the result. After answering each question there is an opportunity to assess its quality and point out errors and inaccuracies. Only when a question is repeatedly checked, and the algorithm can confidently determine its complexity, does it begin to influence the rating.
')
As new questions are added to the test, the rating may decrease over time. At any time, you can answer a few more questions to confirm your level. In addition, the rating is affected by the addition of questions. If you come up with a complex and high-quality question - your rating will grow.

Thanks to this approach, Smarterer managed to achieve a very good level of questions in popular tests. The answer to each question gives the number of points, which depends on the complexity of the question and the level of the subject. The system selects the complexity of the questions in accordance with the result of the test. This ensures fast “convergence” of test results to a real level of knowledge.

We asked Zoya Bogdashkina, head of Luxoft Personnel IT staff selection, to comment on this resource.

“The resource looks quite attractive, I will explain why. Many companies create and conduct professional testing independently, or assign it to agencies that have a set of type tests for various roles and specializations. But it is very important what assessment criteria are determined and how true these criteria are. Of course, a technical expert of each individual company will say that the test he developed is the most correct and reliable.

But, unfortunately, this is not always the case, since the subjective feeling that “this” should be known to all professionals of the desired direction does not always coincide with reality. In our practice, cases happened when, after a while, we understood that candidates did not pass the test, not because their level was bad, but because of insufficiently disclosed information in the test itself. The result - the negotiations, adjustments and hiring a specialist, but with a deadline.

Thus, a valid test can prevent such problems, - says Zoya Bogdashkina , - no less important is the question of the criteria by which candidates should be evaluated. Naturally, each company has its own and it is often difficult to understand how the requirements for a specialist are adequate to the same market. This is especially true for startups, when at the stage of team building, it is not always the project leaders who are well aware of the technical side of the issue and have those. a specialist in a team, which means that they must either involve outside experts or rely on luck. And in fact, and in another case will have to pay. In the latter - the price can be very high, because incorrect technical implementation may entail a number of difficulties. That is why the ability to use such tests, tested by a large audience, looks very tempting.
The very idea of ​​crowdsourcing is now extremely popular and in demand, because it does not allow the product (in this case, the test) to stagnate, it is constantly being finalized.

Personally, I would be interested, on the basis of the ratings already received, to understand the level of technical education of today's specialists in principle and identify trends. ”