Improving the quality of staff selection based on data
Over the past few years, I manage the development and I regularly have to recruit new employees.
Looking at how things are on average in the IT industry, I dare to give a rather negative assessment: in my opinion, interviews are full of subjectivity and randomness, and the average quality of the selection is mediocre - employers complain about the inadequacy of candidates' requests, vacancies can remain months unaccounted for, and staff members often fail to live up to expectations.
I suppose that the reason is the fact that very few of the techies conducting the interviews have an education in the field of personnel management (of course), or at least read something about it. And recruiters, in turn, have little understanding of data analysis. As a result, a pair of these competences is rarely combined in one person and those who hire are simply repeating the outward signs of the interviews they like, not understanding what purpose they served initially and what information they were going to extract. As a result, with each such copy-paste, the quality of decision-making falls.
')
Considering my technical specialization, I tried to improve the quality of selection and, at the same time, reduce the time required for this, developing a process based on objective data, and introducing it for hiring developers to my department. As a result, the process has demonstrated efficiency, has spread widely to the companies in which I worked, and is now being used to hire specialists of a wide variety of fields.
A couple of years ago, I already talked about it on HR Unconference . But there are no performances, and acquaintances who cannot find people in the department are more often interested in details, so I decided to finally describe everything in detail, and at the same time publish my first post on Habré, sharing my work with a wide range of readers. .
Here are the key properties of this process:
So, let's start with a typical process and will improve it. Usually this is the case:
Sometimes quite extravagant techniques are used, such as a long conversation on a free topic, the use of intuition in trying to discern hidden talents and even an analysis of a Vkontakte playlist (all these are real cases).
First, let's start improving the text of the vacancy.
There are a dozen “mandatory requirements” in the job description, with a candidate who satisfies only half of them as a result. Common situation? Is it possible in this case to assume that the requirements were really so mandatory?
In my experience, most often this is due to the overestimated expectations of the employer (which, by the way, also leads to the fact that it is not possible to select anyone for several months of interviews), as well as due to the fact that “mandatory requirements” his personal Wishlist (“I will not be able to work with someone who is not familiar with bash, even if it is a tester”; “if the developer has not encountered git, then he never seriously worked and I will not consider him”; ...) . Partly, such requirements arise from the fact that a personal interview takes a lot of time and takes a lot of energy, so you have to look for some heuristics to screen out more dubious candidates at the stage of reviewing a resume.
So what is required to improve the process.
In this step, separate the truly obligatory from the desired. Think, are you ready to take as a last resort a person who will not be able to do this and that from your list of mandatory requirements? If so, transfer such requirements to desirable ones. Do not be afraid that too many unworthy candidates will end up for a personal interview - we will defend ourselves against this in the next step. Now it is important to understand what is necessary and what is not. It is better to fear that because of small discrepancies, really worthy candidates will be screened out.
The following simple checklist will help you to separate the mandatory from the desired:
In general, one should pay attention to the following thought: in any case, any IT specialist will have to learn something. Even if you magically find a person whose profile is 100% compliant with current requirements, in a couple of months the situation will change and there will most likely be a new demand that everyone will not meet. There may be new libraries in the project that need to be explored; You will introduce new technologies, methodologies and employees will have to figure it all out. Therefore, it is much more important for a person to show a good learning curve than to have a high learning value at the moment.
So, after the described manipulations, no more than 3-5 points should remain in the list of mandatory requirements. These are the skills that all employees are required to possess and which while cannot be acquired relatively quickly. Here is an example of what might have happened for a developer job:
Now your task is to check.
A person can not be objective and unbiased in the matter of evaluating candidates, as it is subject to many cognitive impairments .
In this regard, I try to eliminate the human factor and objectify the selection process as much as possible. An additional positive effect is that the process becomes reproducible by any person and the personality factor of the interviewee ceases to influence the outcome of the candidate.
TripleByte , a company specializing in the selection of candidates, presented 10 theses on how to work with candidates and called them their own manifesto . My experience confirms their theses by 100%.
I found several ways to evaluate candidates, which make it possible to objectively compare them according to formal criteria and so that the results of this comparison would be highly correlated with the actual level of the candidate.
To make a good test with answer choices is not easy. In order not to allow cheaters to get the highest marks, the test must meet the following criteria:
Over time, I developed several techniques that help write good questions:
PluralSight (formerly Smarterer) offers adaptive tests with answer choices on a variety of topics - each following question is selected according to the difficulty of the previous answers. In the end, you can pass through several tests on topics that interest you and be inspired by the questions you like to create your own similar ones.
Good examples of questions can also be found at TripleByte , and the text of the questions contains examples of code in completely different programming languages, but you can answer all the questions without knowing these languages. Thus, they check not only ingenuity, but also the candidate's motivation - some people give up, seeing the name of an unfamiliar language, without even trying to read the code. In my experience, people answer either 90 percent or more of such questions, or 50 or less — there are practically no intermediate results.
Blocks of code I, of course, give screenshots to make it difficult to start them.
For testing, I use the TypeForm platform. Among other things, the platform can calculate the total time spent on the test, which is an important factor. The form can be branded in your corporate colors. I also like the SurveyGizmo platform, I used to use it before.
It should be borne in mind that the main task of this stage is to weed out a substantial part of unworthy candidates (and thereby cover more incoming leads), leaving as far as possible all the worthy ones. That is, false positives are not as terrible as false negatives. Therefore, I try to keep the requirements for passing the test a bit low (we reveal strengths, not weaknesses).
Eliminate screening for resumes and send everyone to the test! This does not require any efforts from the company, and the candidate is also pleased with a good test. I often met good candidates with a bad resume and vice versa, and made sure that the ability to beautifully describe their experience is a separate skill that usually weakly correlates with the working qualities of the candidate, so you shouldn’t be hooked and draw any conclusions on the summary .
Many companies offer tasks like “how many manholes in our city” or “how to determine a false coin in N weighings” during interviews, and then they are surprised that the candidates who have answered these questions perfectly (that is, obviously, many in their lives went to interviews) and those who know such tasks by heart) then work poorly. Google, which has long been famous for using similar tasks in its interviews, ultimately acknowledged that they do not work.
I adhere to the following position: in order to understand how a person will work well, one must at an interview ... get him to work!
For this, a test task is perfectly suitable, imitating the usual work activity to the maximum. I develop such tasks for each position. The following is an example for a developer position.
Developers are then given a sequence of tasks. These are not olympiad tasks, although outwardly everything looks like! These are tasks that best reflect what the candidate has to do in the workplace. For example, for candidates for a position that assumed to be connected to a stream of various ad networks by API, we chose an API with which no one worked exactly (to equalize the initial positions of the candidates) and did tasks related to this API, which check:
In general, everything that you really come up with when working with different APIs. All this we managed to find in the above API reference.
A good starting point for those who have no ideas may be CryproPals.com , which contains simple tasks on cryptography, the solution of which does not require any special knowledge, including cryptography.
For admins, we prepared a virtual machine on which we need to sort out the problems and fix the configuration in order to achieve the load target values. The key to success for the candidate in this case is a combination of basic knowledge of basic technologies, the ability to find information on the Internet, analyze logs and use bash effectively. Yandex offers similar tasks within the framework of the Yandex.Root Olympiad.
Testers were given a page and asked to find as many problems as possible. At the same time, problems can be of a very different plan: and bugs when entering boundary values, and vulnerabilities, and the deterioration of responsiveness when filling the system with a large amount of data, and errors in the js-console, and problems with usability. Everything depends only on what competencies we are looking for in the candidate.
With account managers, it was a bit more difficult, but they found a way to form a practical task for them: we gave synthetic cases to customers' objections and asked the manager to write answers, and then noted on points which of the important installations were used in the answers (for example, if It was noted that communication takes place with the old client, generating companies a significant part of the turnover, it was important that the manager tried to find a win-win solution even though the client insists on an option that is unprofitable for himself, but is beneficial om for the company in the short term).
An important positive feature of this assignment is that the candidate does not need to be controlled, because to solve the problem he is allowed to use any means at his disposal, as well as in the normal working process.
A personal interview is the most time-consuming and most subjective, so I try to check only what is impossible to check at earlier stages. Usually, the case remains small:
The latter is checked by additional questions on the passed tests: why did you answer this way; and how the answer will change if this introductory one changes; why in a practical assignment you implemented such a thing in this way.
Of course, such stories do not happen often (it seems that such cases never came across to me), and in any case everything will quickly become obvious on a probationary period, but why bring this up. In the end, sometimes recruiters have to pay for the candidates they have taken, so it’s best to identify the cheaters as soon as possible.
By the way, it was at the beginning of a personal interview that, to the surprise of the candidates, for the first time I open their resume.
You can try other options for objective testing of the qualities of a candidate, the ways described above are not limited to the world. I used the test task for a while, but then decided that this stage was redundant and refused it. Remember that the main idea is not that there is a test and there is a practical task - use them. And in that, in the first step, you should understand what you really need from the candidate, and then find the simplest objective way to check it.
You should not take it on faith that this mechanism will work immediately. So it will not. I use the method of "gradual input" of this process.
At the first stage, I ask you to take a test of people whose level I know well and whose results I can predict. Usually these are current and former colleagues and friends, whose level of professionalism is perfectly clear to me. I let them test each stage and see how predictable their result was for me. You can also try to give, for example, the admin task to the developers and make sure that their result will be lower than the checkpoint, while professional admins will get a passable result. Ideally, we should make the qualifying stages so that sufficiently strong people pass them almost 100% of the time, and the weak do not go as often as possible.
If we succeed in achieving a stable correlation, I turn to the second stage - the combat test. I bring the first 5-10 candidates to a personal interview, regardless of their results at the qualifying stages, and during a personal interview I ask each person present independently of the others to give his or her assessment of the candidate for each of the criteria checked by tests. At the same time, the present colleagues do not know the result of the candidate for the tests, as well as the assessments of each other (in order to minimize the influence of group thinking). If candidates who have not successfully passed the qualifying stages, one after the other will fill up and a personal interview, then you can stop calling them.
If the correlation between the qualifying stages and the result of a personal interview is low, then try to localize the problems and fix them.
In a test with answer choices, do the following:
In a practical assignment, try to eliminate random factors. For example, there are such difficulties, the collision with which does not say anything bad about the candidate, but has a strong negative impact on the result. For example, in a task with an API, such a factor was that the authors of the API offered a library to work with it, which in fact did not work. Those who tried to use it, lost a lot of time and ended up with a bad result. We made a task in the task that we should not use this library. After all, an attempt to use it a priori is not a wrong action, and therefore should not adversely affect the result.
The tests are not always good the first time, but almost always one iteration of improvements is enough for the test and the practical task to start working perfectly.
When the system is established, all interaction at the test stage with response options can be delegated to absolutely any employee of the organization — for example, a personnel manager or an office manager. The second stage can also be carried out by any person, however, in practice, the person is held in the same position that a new employee is looking for in order to answer the candidate's questions about the company and thus motivate him a little for the second stage. Communication with the candidate takes at this stage 5-10 minutes of time.
Delegating, make sure that the objectivity of the selection does not decrease - make it obligatory to fix the formal results for each candidate employed.
When the system is established, I succeed, based on the results of a personal interview, to take more than half of the people who hit it. All candidates that were recruited this way passed the probationary period and I can call them excellent, effective employees.
As soon as there is a need to expand the staff, an effective search for the next new employee with the already prepared above described system becomes very simple and fast: the main thing is to organize the incoming candidate flow, and choosing the best option from this flow becomes a very simple task that is solved as soon as possible and without pain. of choice.
So, I repeat the main steps:
Looking at how things are on average in the IT industry, I dare to give a rather negative assessment: in my opinion, interviews are full of subjectivity and randomness, and the average quality of the selection is mediocre - employers complain about the inadequacy of candidates' requests, vacancies can remain months unaccounted for, and staff members often fail to live up to expectations.
I suppose that the reason is the fact that very few of the techies conducting the interviews have an education in the field of personnel management (of course), or at least read something about it. And recruiters, in turn, have little understanding of data analysis. As a result, a pair of these competences is rarely combined in one person and those who hire are simply repeating the outward signs of the interviews they like, not understanding what purpose they served initially and what information they were going to extract. As a result, with each such copy-paste, the quality of decision-making falls.
')
Considering my technical specialization, I tried to improve the quality of selection and, at the same time, reduce the time required for this, developing a process based on objective data, and introducing it for hiring developers to my department. As a result, the process has demonstrated efficiency, has spread widely to the companies in which I worked, and is now being used to hire specialists of a wide variety of fields.
A couple of years ago, I already talked about it on HR Unconference . But there are no performances, and acquaintances who cannot find people in the department are more often interested in details, so I decided to finally describe everything in detail, and at the same time publish my first post on Habré, sharing my work with a wide range of readers. .
Here are the key properties of this process:
- it is best suited for vacancies for which new employees are hired regularly, as it requires spending some time on adjustment, which, however, very quickly pays for itself when regularly hired, turning decision making for each candidate from a creative process into a routine that takes a few minutes for a candidate and while providing higher quality solutions;
- At the same time, the percentage of accepting offers by candidates rises, since, firstly, the selection process itself makes a positive impression on them, and secondly, using this process, it is possible to get ahead of all competitors and offer an offer to a person literally on the same day when he responded to the vacancy.
So, let's start with a typical process and will improve it. Usually this is the case:
- vacancy overloaded with “mandatory requirements”;
- the first stage is screening by resume;
- then, perhaps, a conversation with “Euchar” for primary screening;
- after that, a lengthy technical interview without a clear plan with a discussion of work experience, deepening into arbitrary details and “puzzling tasks”, which resulted in a final decision.
Sometimes quite extravagant techniques are used, such as a long conversation on a free topic, the use of intuition in trying to discern hidden talents and even an analysis of a Vkontakte playlist (all these are real cases).
Step 1. Sift the job
First, let's start improving the text of the vacancy.
There are a dozen “mandatory requirements” in the job description, with a candidate who satisfies only half of them as a result. Common situation? Is it possible in this case to assume that the requirements were really so mandatory?
In my experience, most often this is due to the overestimated expectations of the employer (which, by the way, also leads to the fact that it is not possible to select anyone for several months of interviews), as well as due to the fact that “mandatory requirements” his personal Wishlist (“I will not be able to work with someone who is not familiar with bash, even if it is a tester”; “if the developer has not encountered git, then he never seriously worked and I will not consider him”; ...) . Partly, such requirements arise from the fact that a personal interview takes a lot of time and takes a lot of energy, so you have to look for some heuristics to screen out more dubious candidates at the stage of reviewing a resume.
So what is required to improve the process.
In this step, separate the truly obligatory from the desired. Think, are you ready to take as a last resort a person who will not be able to do this and that from your list of mandatory requirements? If so, transfer such requirements to desirable ones. Do not be afraid that too many unworthy candidates will end up for a personal interview - we will defend ourselves against this in the next step. Now it is important to understand what is necessary and what is not. It is better to fear that because of small discrepancies, really worthy candidates will be screened out.
The following simple checklist will help you to separate the mandatory from the desired:
- Do all employees need this skill? For example, some of the system tasks are related to the revision of the internal admin that uses Vue.js. If only half of the development staff know Vue.js, will their resources be enough to solve all the tasks of such a plan?
- Can a person quickly learn this skill? For example, if work in your department is built on git flow, it is not necessary to demand this skill from candidates - any person can easily figure out the git flow in a couple of first working days, even if you have never worked with git before.
In general, one should pay attention to the following thought: in any case, any IT specialist will have to learn something. Even if you magically find a person whose profile is 100% compliant with current requirements, in a couple of months the situation will change and there will most likely be a new demand that everyone will not meet. There may be new libraries in the project that need to be explored; You will introduce new technologies, methodologies and employees will have to figure it all out. Therefore, it is much more important for a person to show a good learning curve than to have a high learning value at the moment.
So, after the described manipulations, no more than 3-5 points should remain in the list of mandatory requirements. These are the skills that all employees are required to possess and which while cannot be acquired relatively quickly. Here is an example of what might have happened for a developer job:
- confident knowledge of the main programming language used in the project (indeed, you will not have time to wait for the employee to master Java better);
- confident knowledge of the key technology used in the project, the development of which is impossible in a short time (for example, multi-threaded programming);
- high speed of development ( they say that the speed of implementation of tasks for different employees may differ by 10 times, and I believe that this indicator is unlikely to vary greatly with time for an individual person);
- high learning ability (as I wrote above, I consider a good learning curve to be the most important indicator);
- compatibility of personal qualities with current employees (even with other advantages, I think that you should not take a person who will not fit into the team).
Now your task is to check.
Step 2. Develop an objective verification mechanism for mandatory requirements.
A person can not be objective and unbiased in the matter of evaluating candidates, as it is subject to many cognitive impairments .
Here are some of them.
The list goes on and on ...
- irrational reinforcement - the more effort was put into the candidate's assessment, the greater the desire to take it so that these efforts are not in vain;
- the effect of the first impression - regardless of the duration of the interview, in most cases, the final decision is subconsciously made in the first 15 minutes of the conversation;
- group thinking - although it seems that a candidate’s assessment by a group of interviewers should be more objective than a single person’s decision making, in fact, groups of 3-5 people most often show conformity with respect to authority, self-censorship of opinions that are different from those of the majority and reduced self-criticism of authority in the case when it is supported by the group. The combination of these factors ultimately leads to the fact that the decisions made by the group are less often qualitative;
- the effect of overconfidence is the revaluation by any person of his own expertise in the ability to distinguish good candidates from bad ones;
- propensity to confirm his point of view - a tendency to pull out from the information coming from the candidate, the kind that confirms the impression formed about him and ignore the fact that he is against him;
- the effect of acquaintance with the object is the tendency to empathize with the candidate as the duration of the meeting increases;
- halo effect - the impression that people with more attractive looks have more outstanding mental abilities;
- in a state of hunger (before dinner) people are much more prone to criticism;
- people are afraid to change and seek to defend their opinion if it has been publicly voiced.
The list goes on and on ...
In this regard, I try to eliminate the human factor and objectify the selection process as much as possible. An additional positive effect is that the process becomes reproducible by any person and the personality factor of the interviewee ceases to influence the outcome of the candidate.
TripleByte , a company specializing in the selection of candidates, presented 10 theses on how to work with candidates and called them their own manifesto . My experience confirms their theses by 100%.
Therefore, below I will give the most relevant to our topic in Russian translation.
- the recruitment process should be standardized. Writing programs on the board and algorithmic questions cannot qualitatively predict how effectively a given person will write real code. Such a technical hiring process harms both excellent candidates who cannot cope well with such tasks, and companies that would like to hire good programmers;
- hiring decisions must be objective. Decisions on hiring should be made using a clear scoring system, not intuitively. People gather information well, but they are too biased. We are unconsciously trying to stick labels. This is detrimental to candidates who do not meet our expectations;
- the recruitment process should focus on identifying strengths, not weaknesses. Everyone has flaws. But the role is played primarily by the virtues that distinguish the candidate from the rest, and how quickly they can learn new things;
- Candidates should know what to expect at the interview. All candidates must have equal opportunity to prepare in advance. A comfortable interview with a candidate is likely to lead to a better hiring decision;
- candidates deserve feedback. Candidates should be given clear, truthful feedback on how they coped with the assignments at the interview and where their growth zones are located.
- diploma means nothing. There should be some text about it, but on the company's website in this place there is a paragraph from another paragraph by mistake. I suppose they wanted to write about the fact that it is impossible to judge unequivocally from the piece of paper about the candidate’s professionalism;
- the recruitment process should be continuously measured and improved. The recruitment process should be viewed as a software product that is constantly improving over time based on data and feedback. You need to experiment more and find out what really works.
I found several ways to evaluate candidates, which make it possible to objectively compare them according to formal criteria and so that the results of this comparison would be highly correlated with the actual level of the candidate.
1. Test with answer options
To make a good test with answer choices is not easy. In order not to allow cheaters to get the highest marks, the test must meet the following criteria:
- the answers should not be google (at least in the allotted time) - for this a good question should not require actual knowledge, but some manipulation in the head, based on a deep understanding of the topic, in order to get the correct answer;
- if the question contains a block of code, it must be impossible to start it in order to know the correct answer;
- the question should not be too simple (in this case, the answer to it will not give any information);
- the question should not be too complicated (if the percentage of correct answers is close to
1 / _
- it should not be checked by knowledge of the fact that one can easily google during work - the presence of this knowledge in the head does not show anything; the correct answer should show understanding, not knowledge;
- what should not be used by the candidate in our work (“general erudition”, clever data structures and algorithms, ability to model physical processes, ...) should not be checked - questions should be compiled only according to mandatory requirements;
- A good number of response options will be from 4 to 7 - with 3 or fewer response options, the probability of guessing is too high, while with more than 7 response options, it is likely that you will not get all the options in your head and get confused.
Over time, I developed several techniques that help write good questions:
- several abstracts (reformulated so as not to google one-on-one), one of them is false (or one of them is correct) - you need to name it;
- find the error in the code / configuration (a number of controversial points, one of which is an error, and the rest are relatively rare cases);
- code / request / ... and the response options that he does. To prevent the launch from being checked, the code can take several arguments as input and not work without them; require a different “strapping”, without which it will not start or will not work correctly;
- code / request / ..., in which the line is missing and options for the code to insert. Substitute the correct version so that the code solves the task specified in the task.
PluralSight (formerly Smarterer) offers adaptive tests with answer choices on a variety of topics - each following question is selected according to the difficulty of the previous answers. In the end, you can pass through several tests on topics that interest you and be inspired by the questions you like to create your own similar ones.
Good examples of questions can also be found at TripleByte , and the text of the questions contains examples of code in completely different programming languages, but you can answer all the questions without knowing these languages. Thus, they check not only ingenuity, but also the candidate's motivation - some people give up, seeing the name of an unfamiliar language, without even trying to read the code. In my experience, people answer either 90 percent or more of such questions, or 50 or less — there are practically no intermediate results.
Here is an example of a good question in my opinion.
It is easy to answer this question without even knowing the JavaScript language. This question is good because it is not enough to type and run this code in order to find out the answer - it is necessary to substitute input data that will unambiguously classify the result, as well as to sort through each of the answer options. It is hardly easier than to think with your head and logically think of the correct answer.
Choose which line variant should be inserted into the given code:
// function minIndex(array) { minIndex = -1; minValue = 0; for (i = 0; i < array.length; ++i) { if (array[i] < minValue || minIndex == -1) { minIndex = i; // } } return minIndex; }
- return minIndex;
- return minValue;
- minValue = array [i];
- i = minValue;
It is easy to answer this question without even knowing the JavaScript language. This question is good because it is not enough to type and run this code in order to find out the answer - it is necessary to substitute input data that will unambiguously classify the result, as well as to sort through each of the answer options. It is hardly easier than to think with your head and logically think of the correct answer.
Blocks of code I, of course, give screenshots to make it difficult to start them.
For testing, I use the TypeForm platform. Among other things, the platform can calculate the total time spent on the test, which is an important factor. The form can be branded in your corporate colors. I also like the SurveyGizmo platform, I used to use it before.
It should be borne in mind that the main task of this stage is to weed out a substantial part of unworthy candidates (and thereby cover more incoming leads), leaving as far as possible all the worthy ones. That is, false positives are not as terrible as false negatives. Therefore, I try to keep the requirements for passing the test a bit low (we reveal strengths, not weaknesses).
Eliminate screening for resumes and send everyone to the test! This does not require any efforts from the company, and the candidate is also pleased with a good test. I often met good candidates with a bad resume and vice versa, and made sure that the ability to beautifully describe their experience is a separate skill that usually weakly correlates with the working qualities of the candidate, so you shouldn’t be hooked and draw any conclusions on the summary .
2. Practical task
Many companies offer tasks like “how many manholes in our city” or “how to determine a false coin in N weighings” during interviews, and then they are surprised that the candidates who have answered these questions perfectly (that is, obviously, many in their lives went to interviews) and those who know such tasks by heart) then work poorly. Google, which has long been famous for using similar tasks in its interviews, ultimately acknowledged that they do not work.
I adhere to the following position: in order to understand how a person will work well, one must at an interview ... get him to work!
For this, a test task is perfectly suitable, imitating the usual work activity to the maximum. I develop such tasks for each position. The following is an example for a developer position.
This is the introductory message I send to candidates in advance so that they are ready for what awaits them.
At the next interview (duration 1.5 hours) you will have to solve simple tasks. Tasks are connected in a single outline - when solving each next task, it will be convenient to build on the code written when solving the previous ones.
By the time this phase begins, I ask you to prepare an environment on your computer in which it will be convenient for you to write code and run it.
But such an introductory, I send them immediately before the start of the interview.
Today's interview will last no more than an hour and a half, during which you will solve problems, one by one.
Before you begin, you need to make sure that your favorite development tool (IDE) is installed and conveniently configured on your computer, you can write and run code in the programming language <...>.
Will be evaluated:
1. The number of tasks that you will be able to solve in the allotted time.
2. The number of attempts that you need to solve each of the tasks.
This stage of the interview is designed to maximize the usual workflow. This is facilitated by the following features:
1. The quality of the code will not be assessed, however, each next task is some development or complication of the previous one. Thus, it is beneficial to write the code so that you can reuse it. As in the normal workflow, you do not know in advance which parts of the code you will have to develop in the future.
2. As at work, you can use any information from the Internet without restriction.
3. Before the “release” (that is, before generating the answer for each of the tasks), you need to make sure yourself that your code works, because in real work no one will find an error for you.
4. Write in a concentrated way, but without haste. By experience, excessive haste compared to your normal working pace will hurt you more than help.
Good luck!
Developers are then given a sequence of tasks. These are not olympiad tasks, although outwardly everything looks like! These are tasks that best reflect what the candidate has to do in the workplace. For example, for candidates for a position that assumed to be connected to a stream of various ad networks by API, we chose an API with which no one worked exactly (to equalize the initial positions of the candidates) and did tasks related to this API, which check:
- candidate care;
- the ability to parse complex answers;
- ability to understand poorly written specifications;
- the ability to detect that the API does not always work correctly and apply workaround;
- the ability to search for additional information on the Internet;
- ability to operate with complex data structures.
In general, everything that you really come up with when working with different APIs. All this we managed to find in the above API reference.
A good starting point for those who have no ideas may be CryproPals.com , which contains simple tasks on cryptography, the solution of which does not require any special knowledge, including cryptography.
For admins, we prepared a virtual machine on which we need to sort out the problems and fix the configuration in order to achieve the load target values. The key to success for the candidate in this case is a combination of basic knowledge of basic technologies, the ability to find information on the Internet, analyze logs and use bash effectively. Yandex offers similar tasks within the framework of the Yandex.Root Olympiad.
Testers were given a page and asked to find as many problems as possible. At the same time, problems can be of a very different plan: and bugs when entering boundary values, and vulnerabilities, and the deterioration of responsiveness when filling the system with a large amount of data, and errors in the js-console, and problems with usability. Everything depends only on what competencies we are looking for in the candidate.
With account managers, it was a bit more difficult, but they found a way to form a practical task for them: we gave synthetic cases to customers' objections and asked the manager to write answers, and then noted on points which of the important installations were used in the answers (for example, if It was noted that communication takes place with the old client, generating companies a significant part of the turnover, it was important that the manager tried to find a win-win solution even though the client insists on an option that is unprofitable for himself, but is beneficial om for the company in the short term).
An important positive feature of this assignment is that the candidate does not need to be controlled, because to solve the problem he is allowed to use any means at his disposal, as well as in the normal working process.
3. Personal interview
A personal interview is the most time-consuming and most subjective, so I try to check only what is impossible to check at earlier stages. Usually, the case remains small:
- to verify the personal compatibility of the candidate with the current employees;
- make sure that the elder brother did not pass the tests for the candidate.
The latter is checked by additional questions on the passed tests: why did you answer this way; and how the answer will change if this introductory one changes; why in a practical assignment you implemented such a thing in this way.
Of course, such stories do not happen often (it seems that such cases never came across to me), and in any case everything will quickly become obvious on a probationary period, but why bring this up. In the end, sometimes recruiters have to pay for the candidates they have taken, so it’s best to identify the cheaters as soon as possible.
By the way, it was at the beginning of a personal interview that, to the surprise of the candidates, for the first time I open their resume.
4. Other options
You can try other options for objective testing of the qualities of a candidate, the ways described above are not limited to the world. I used the test task for a while, but then decided that this stage was redundant and refused it. Remember that the main idea is not that there is a test and there is a practical task - use them. And in that, in the first step, you should understand what you really need from the candidate, and then find the simplest objective way to check it.
Step 3. Tuning
You should not take it on faith that this mechanism will work immediately. So it will not. I use the method of "gradual input" of this process.
At the first stage, I ask you to take a test of people whose level I know well and whose results I can predict. Usually these are current and former colleagues and friends, whose level of professionalism is perfectly clear to me. I let them test each stage and see how predictable their result was for me. You can also try to give, for example, the admin task to the developers and make sure that their result will be lower than the checkpoint, while professional admins will get a passable result. Ideally, we should make the qualifying stages so that sufficiently strong people pass them almost 100% of the time, and the weak do not go as often as possible.
If we succeed in achieving a stable correlation, I turn to the second stage - the combat test. I bring the first 5-10 candidates to a personal interview, regardless of their results at the qualifying stages, and during a personal interview I ask each person present independently of the others to give his or her assessment of the candidate for each of the criteria checked by tests. At the same time, the present colleagues do not know the result of the candidate for the tests, as well as the assessments of each other (in order to minimize the influence of group thinking). If candidates who have not successfully passed the qualifying stages, one after the other will fill up and a personal interview, then you can stop calling them.
If the correlation between the qualifying stages and the result of a personal interview is low, then try to localize the problems and fix them.
In a test with answer choices, do the following:
- find questions that everyone answers. Eliminate them or make it more difficult, otherwise they will not carry any information;
- Find questions that are answered approximately at the level of a random hit in the answer option. Eliminate or simplify them. The question gives the most information if it answers about 50-60% of the candidates;
- see if the correct answers are not burning according to some simple criterion. For example, how many points will a person get in your test if he always chooses the longest answers? Most likely, if you have not thought about this before, then most of the correct answers will be the longest;
- calculate the correlation of each individual question with the final impression from the candidate and with the result of passing through other stages (the correlation calculation function is available out of the box in any spreadsheet). Questions that have a close to zero or negative correlation should be removed or improved - most likely they have some problem and they check not what is needed;
- if a strong person answered the question incorrectly - specify on a personal interview why he chose this option. Maybe you didn’t take something into account, the question was compiled with an error, contains ambiguity, or simply does not check anything;
- instead of all deleted questions, invent new ones in order to keep their number approximately.
In a practical assignment, try to eliminate random factors. For example, there are such difficulties, the collision with which does not say anything bad about the candidate, but has a strong negative impact on the result. For example, in a task with an API, such a factor was that the authors of the API offered a library to work with it, which in fact did not work. Those who tried to use it, lost a lot of time and ended up with a bad result. We made a task in the task that we should not use this library. After all, an attempt to use it a priori is not a wrong action, and therefore should not adversely affect the result.
The tests are not always good the first time, but almost always one iteration of improvements is enough for the test and the practical task to start working perfectly.
Step 4. Delegate
When the system is established, all interaction at the test stage with response options can be delegated to absolutely any employee of the organization — for example, a personnel manager or an office manager. The second stage can also be carried out by any person, however, in practice, the person is held in the same position that a new employee is looking for in order to answer the candidate's questions about the company and thus motivate him a little for the second stage. Communication with the candidate takes at this stage 5-10 minutes of time.
Delegating, make sure that the objectivity of the selection does not decrease - make it obligatory to fix the formal results for each candidate employed.
Total
When the system is established, I succeed, based on the results of a personal interview, to take more than half of the people who hit it. All candidates that were recruited this way passed the probationary period and I can call them excellent, effective employees.
As soon as there is a need to expand the staff, an effective search for the next new employee with the already prepared above described system becomes very simple and fast: the main thing is to organize the incoming candidate flow, and choosing the best option from this flow becomes a very simple task that is solved as soon as possible and without pain. of choice.
So, I repeat the main steps:
- look critically at the list of mandatory requirements and try to keep only a few main ones;
- , — , ; , ;
- , ;
- , .
Source: https://habr.com/ru/post/341220/
All Articles