Redefining the interview process in the Microsoft Development Division
A couple of years ago I had a series of insights. I spoke with my team about how we are going to change the position of the program manager. For example, less attention is given to backlog, and more to business; to pay less attention to “knowledge” and more to “training and questions”; focus more on customer interaction 1: 1, and less on aggregated data. We wanted to bring in a team of people who would help us change this culture, but we still asked the same questions at the interviews, and the interviews themselves were all in the same style. Therefore, we rethought our interview process.
We have been doing this for some time and now we want to tell what we have done and what we have learned.
')
This understanding came after the transition to the post of program manager: we noticed that we were asking the same questions at the interview that we had asked for the last ten years or more. These questions did not make sense when we tried to find people who could bring different skills and points of view to the team. I started working at Microsoft when we still asked questions about why the manhole covers are round, how many ping-pong balls you need to fill the Boeing 747, and how to reverse the linked list. For 20 years here I have never had to write code to invert a coherent list or fill the plane with some kind of balls.
In addition, sometimes two personnel officers unintentionally asked the same basic question. Even when we agreed on the division of topics for interviews, we relied on the same basic catalog of questions. Some behavioral issues were not terrible, but we didn’t use them very effectively.
And this understanding came to me during one of the gliders. Ideas arose one after another, people eagerly began to speak, not waiting for others to finish, in order to have time to talk about their ideas, and we quickly went to a fateful decision. Well, I am sure that it had to be fateful - I was distracted by the problem with the client, and I corresponded with the account manager and the team of engineers to resolve the situation. But that's another story. In short, the meeting was getting more and more emotional, and one of the participants, the program manager on my team, is very clever and silent, - in fact, he said: “I just searched the network for information on our topic, and yes, this idea will not work.” She at the time expressed softer. But I realized that not everyone works best during such active brainstorming. Many, including me, prefer to sit with a cup of coffee and deal with some data. Moreover, in my memory there were almost no cases when we would have made an important decision without having to step back and evaluate the idea with a fresh mind, with fresh data and fresh client research.
However, this was the way most interviews took place: intensively, as-quickly-you-can-find-solutions-problems-with-which-earlier-not-encountered.
Understanding covered me when I talked with a couple of our engineering teams about how they attract people to themselves. The development division does a lot for open source traffic (.NET Core, VS Code, TypeScript and many other projects). Our developers, along with the candidates, worked on fixing bugs or implementing features, using it as part of the interview process.
Guided by the principle “a letter is a reflection,” I wrote an e-mail myself about how different interviews can take on my team. Then I shared my ideas with many colleagues, and we started testing them over and over again. This included Karen Ng , Amanda Silver , Cindy Alvarez , Nathan Halstead , Anthony Cangialosi , Jeff McAffer , Jessica Rich, Travis Lowdermilk and many others.
And when we were ready to apply the resulting ideas in practice, we started small and continued to learn, iterated and expanded the methodology. Now this framework (we called it the “alternative interview framework”, because none of us have the gift of giving good names) has become a standard tool that works great for our tasks, which we continue to improve and on which we learn.
That's what we changed.
To begin with, tell the candidate a few days before the interview how it will go and what task you will be working on. We give people time to search for solutions and thoughts on their own. After all, when we come to work, it does not turn into daily surprises, so why should the interview be held in this way?
We discuss the real tasks that the team is trying to solve: we improve user experience of products, increase retention, encourage the use of a service or function. The fact that this is a real problem we are working on helps to develop conversation.
We give candidates access to the same information that we work with ourselves, and during the interview people can freely use the Internet or request other data. We often give candidates our client research, product usage information, design solutions and stubs - most of what we have.
We conduct interviews in an interactive form. We do not ask questions. We need to jointly solve some problem, so let's do it as if you worked with us and together we solved this problem.
During the day, we follow only one scenario or solve only one task, guide the candidate along the same path that program managers take: we start by analyzing customer or business problems, then we find out what tasks we need to solve, design solutions, and finally, deliver its in the hands of customers. As a result, you need to make customers use and love this decision. Each interview is devoted to different stages of this process.
Instead of interviewing candidates one on one, we give two people from each team for each interview. Initially, we began to do this in order to train more colleagues for interviews, but it turned out that there were other advantages to this move. The conversation is more dynamic, and at the same meeting we can hear several points of view. People can perceive the same words differently, so that in the course of a single conversation we can identify unconscious prejudices.
Interviewers do not share their opinions until the end of the day. We wanted each employee to evaluate candidates independently, without taking into account the opinion of colleagues.
We ask the interviewees not to give each other signals about whether to hire one or another person. They pass the candidate to another pair of interviewers and write a review of their conversation. At the end of the day, everyone gives their recommendations and small explanations about what they saw and heard, which led them to a specific conclusion.
At the end of each interview cycle, we discuss not only what we learned after conversations with the candidate, but also what approaches worked or did not work during the interviews. And on the basis of this information we improve the process itself.
Surely I forgot about something, but these eight aspects are the most important.
We thought the candidates would be nervous. Two interviewers; the real problem that we solve during the day with real data — we should not have done so. However, almost all candidates voluntarily said that this interview procedure was unusual and helped them understand our company and team. Even those who did not receive suggestions about the device liked the interview process, and they understood why we didn’t take them.
We found out that there are some gaps. For example, our program managers turned out to be exclusively technicians. Many of them check the code in the finished products. For us, it makes sense: our customers are developers, so when creating software it is important to understand your client. But in the interview process we did not provide for a thorough verification of the technical skills of the candidates, so we added another step to this.
It turned out that the standard “logistics” of interviews is inconvenient for our new process. For example, if a candidate works on one problem and writes something on the board that he will need later, then you need to allocate a separate room for the candidate (a conference room or a focus room), and the interviewers will come to him.
Citing two employees for interviews, we double the loss of man-hours and workers during the day, and it also makes it very difficult to agree on work schedules. But after we explained to the staff the “high cost” of interviews, everyone realized the benefits of increasing the number of interviewers. So we were ready to pay our price for it.
The purpose of the interviews is to find great people for a team or company, to make sure that they are suitable for us and will work successfully, and also to make them want to come to us. One of the candidates who received a job offer from us and a couple of other major technology companies from Seattle chose our team because she liked the interview process. She was one of the first people we interviewed during the experiments, and she still works with us, like many other employees who came after her. So I think we did it well - we are still learning, but the result has already greatly exceeded our expectations.
We have been doing this for some time and now we want to tell what we have done and what we have learned.
')
Insight # 1: We are still asking outdated (and ineffective) questions.
This understanding came after the transition to the post of program manager: we noticed that we were asking the same questions at the interview that we had asked for the last ten years or more. These questions did not make sense when we tried to find people who could bring different skills and points of view to the team. I started working at Microsoft when we still asked questions about why the manhole covers are round, how many ping-pong balls you need to fill the Boeing 747, and how to reverse the linked list. For 20 years here I have never had to write code to invert a coherent list or fill the plane with some kind of balls.
In addition, sometimes two personnel officers unintentionally asked the same basic question. Even when we agreed on the division of topics for interviews, we relied on the same basic catalog of questions. Some behavioral issues were not terrible, but we didn’t use them very effectively.
Insight number 2: not everyone is able to work as efficiently as possible at a frantic pace and when you are under heavy pressure
And this understanding came to me during one of the gliders. Ideas arose one after another, people eagerly began to speak, not waiting for others to finish, in order to have time to talk about their ideas, and we quickly went to a fateful decision. Well, I am sure that it had to be fateful - I was distracted by the problem with the client, and I corresponded with the account manager and the team of engineers to resolve the situation. But that's another story. In short, the meeting was getting more and more emotional, and one of the participants, the program manager on my team, is very clever and silent, - in fact, he said: “I just searched the network for information on our topic, and yes, this idea will not work.” She at the time expressed softer. But I realized that not everyone works best during such active brainstorming. Many, including me, prefer to sit with a cup of coffee and deal with some data. Moreover, in my memory there were almost no cases when we would have made an important decision without having to step back and evaluate the idea with a fresh mind, with fresh data and fresh client research.
However, this was the way most interviews took place: intensively, as-quickly-you-can-find-solutions-problems-with-which-earlier-not-encountered.
Insight # 3: The best way to understand how someone works is to work with him.
Understanding covered me when I talked with a couple of our engineering teams about how they attract people to themselves. The development division does a lot for open source traffic (.NET Core, VS Code, TypeScript and many other projects). Our developers, along with the candidates, worked on fixing bugs or implementing features, using it as part of the interview process.
We write down idea and we iterate it
Guided by the principle “a letter is a reflection,” I wrote an e-mail myself about how different interviews can take on my team. Then I shared my ideas with many colleagues, and we started testing them over and over again. This included Karen Ng , Amanda Silver , Cindy Alvarez , Nathan Halstead , Anthony Cangialosi , Jeff McAffer , Jessica Rich, Travis Lowdermilk and many others.
And when we were ready to apply the resulting ideas in practice, we started small and continued to learn, iterated and expanded the methodology. Now this framework (we called it the “alternative interview framework”, because none of us have the gift of giving good names) has become a standard tool that works great for our tasks, which we continue to improve and on which we learn.
That's what we changed.
We talk in advance about the interview
To begin with, tell the candidate a few days before the interview how it will go and what task you will be working on. We give people time to search for solutions and thoughts on their own. After all, when we come to work, it does not turn into daily surprises, so why should the interview be held in this way?
We use real tasks
We discuss the real tasks that the team is trying to solve: we improve user experience of products, increase retention, encourage the use of a service or function. The fact that this is a real problem we are working on helps to develop conversation.
We give access to data
We give candidates access to the same information that we work with ourselves, and during the interview people can freely use the Internet or request other data. We often give candidates our client research, product usage information, design solutions and stubs - most of what we have.
We make interviews interactive
We conduct interviews in an interactive form. We do not ask questions. We need to jointly solve some problem, so let's do it as if you worked with us and together we solved this problem.
Follow the same scenario.
During the day, we follow only one scenario or solve only one task, guide the candidate along the same path that program managers take: we start by analyzing customer or business problems, then we find out what tasks we need to solve, design solutions, and finally, deliver its in the hands of customers. As a result, you need to make customers use and love this decision. Each interview is devoted to different stages of this process.
We are interviewing in pairs
Instead of interviewing candidates one on one, we give two people from each team for each interview. Initially, we began to do this in order to train more colleagues for interviews, but it turned out that there were other advantages to this move. The conversation is more dynamic, and at the same meeting we can hear several points of view. People can perceive the same words differently, so that in the course of a single conversation we can identify unconscious prejudices.
Do not discuss the candidate until the end of the day
Interviewers do not share their opinions until the end of the day. We wanted each employee to evaluate candidates independently, without taking into account the opinion of colleagues.
We ask the interviewees not to give each other signals about whether to hire one or another person. They pass the candidate to another pair of interviewers and write a review of their conversation. At the end of the day, everyone gives their recommendations and small explanations about what they saw and heard, which led them to a specific conclusion.
Give feedback and about the process
At the end of each interview cycle, we discuss not only what we learned after conversations with the candidate, but also what approaches worked or did not work during the interviews. And on the basis of this information we improve the process itself.
Surely I forgot about something, but these eight aspects are the most important.
What have we learned?
We thought the candidates would be nervous. Two interviewers; the real problem that we solve during the day with real data — we should not have done so. However, almost all candidates voluntarily said that this interview procedure was unusual and helped them understand our company and team. Even those who did not receive suggestions about the device liked the interview process, and they understood why we didn’t take them.
We continue to improve the process
We found out that there are some gaps. For example, our program managers turned out to be exclusively technicians. Many of them check the code in the finished products. For us, it makes sense: our customers are developers, so when creating software it is important to understand your client. But in the interview process we did not provide for a thorough verification of the technical skills of the candidates, so we added another step to this.
Logistics
It turned out that the standard “logistics” of interviews is inconvenient for our new process. For example, if a candidate works on one problem and writes something on the board that he will need later, then you need to allocate a separate room for the candidate (a conference room or a focus room), and the interviewers will come to him.
Interview "expensive"
Citing two employees for interviews, we double the loss of man-hours and workers during the day, and it also makes it very difficult to agree on work schedules. But after we explained to the staff the “high cost” of interviews, everyone realized the benefits of increasing the number of interviewers. So we were ready to pay our price for it.
The purpose of the interviews is to find great people for a team or company, to make sure that they are suitable for us and will work successfully, and also to make them want to come to us. One of the candidates who received a job offer from us and a couple of other major technology companies from Seattle chose our team because she liked the interview process. She was one of the first people we interviewed during the experiments, and she still works with us, like many other employees who came after her. So I think we did it well - we are still learning, but the result has already greatly exceeded our expectations.
Source: https://habr.com/ru/post/459392/
All Articles