DataLine experience: how we prepare duty engineers for our data centers

Today we want to tell you how we select and prepare duty engineers for our data centers. This position is important for us for two reasons:

• This is our first line of technical support. Attendants monitor the monitoring wall, communicate with customers who come to our data centers, help to install new equipment. Their efficient work, friendly attitude to customers, attention to trifles mean for our service no less than the work of the infrastructure of the data center: exceeded the response time to the request, allowed to go to the data center without shoe covers, just did not say hello - minus our karma
  
  
• this is our “forge of personnel”. Some guys then grow to lead engineers of specialized technical groups and even to department heads.

But first things first.

What do the engineers on duty

The engineers on duty have many tasks. They work:

• with clients: accompany them in the data center, help clients arrange deposit / removal of equipment, monitor compliance with the rules of behavior in the data center.
• with IT equipment: mark and register in the accounting system, prepare racks for installation, assemble and disassemble, reboot it, lay cross-connects.
• with engineering infrastructure: twice a day, visually inspect all the infrastructure facilities of the data center.
• with monitoring system: track alerts, record, resolve and, if necessary, escalate incidents.

In the day shift, depending on the data center, 6-8 engineers on duty and one senior engineer are on duty. In the night - two people less.
Engineers work on two schedules to choose from: 5 days a week for 8 hours or a day after three.

The posts of engineers on duty are in the monitoring center. The work is structured so that at least one person always follows the monitoring wall.

Post duty engineers of some of our data centers. In total, the OST site employs 19 people on duty, NORD - 27 people.

Who can become a duty engineer

We prefer graduates and students of the last courses of technical universities. Many come from the OST MTUCI located next to the site, MSTU. Bauman, MEI.

Duty engineer - starting position. In our experience, after a year of work, the engineer gets all the possible knowledge in this position, and in the “duty room” he becomes bored. At the same time, by this point he understands where to go next.

If the engineer proved himself well, possesses the necessary knowledge, and we have vacancies, then in a year or even earlier the engineer on duty can try himself in more senior positions. It has the following options:

• stay on duty as a senior shift engineer with the possibility of becoming a senior on the site.

  
  

• continue to work in the second line of support - specialized technical departments: maintenance of the engineering infrastructure, virtualization, network, Unix, Windows-groups.

  
  
• go into business areas - directorate of sales and service management, product development department.

Many of the current leading engineers and even heads of departments began their work in the “duty room”. Growing up with desire is possible, and we only welcome it. Those who have gone this way are especially valuable because they know everything about operational processes and the design of our data centers. Here are a couple of such stories.

How are the selection and internship

From the moment of sending a resume to enrollment, it takes from 1 to 3 months, depending on starting knowledge and skills. During this time, both we and the candidate himself will understand whether we are suitable for each other.
The candidate’s path to the rank of “duty engineer” is as follows:

qualifying round - 1 day;
introductory study and exam - 3 days;
paid internship - 1-3 months;
enrollment in the state.

Qualifying round. All whose resumes seemed interesting to us are invited to the qualifying round, which is conducted by experts from the HR-department. At this stage, we meet with the candidate and acquaint him with our company, future responsibilities, opportunities, conditions for salary. No less detailed and honestly talk about the difficulties of the position, so that the future engineer understands what he will have to face. After a short story about themselves, candidates are tested in combat conditions: they decide cases that are as close to reality as possible. Here is an example:

A service manager is writing you a letter asking you to connect an additional rack for the client. There is no official confirmation from your manual. Today is Friday 7:00 pm, your shift ends at 9 am on Saturday, and work must be delivered at 12 am on Saturday. Your actions?

From the candidate is not required unambiguously correct answer. Moreover, it is proposed to solve the problem jointly. At this stage, it is important for us to understand how he is guided when deciding whether he has enough self-control (is it not rude, not falling into a stupor). Another important point for us is the ability to work with other people. There are always several people on duty in the shift, so it is important for us to understand how quickly and well the candidate finds a common language with his colleagues.

Together with the invitation to the qualifying round, the guys receive instructions on the simplest and most frequent operations in the data center: compress the twisted pair, configure the IP KVM, remove logs from the PDU, etc. Understand them and understand what's what - a kind of homework. At the interview you need to do everything yourself, if possible without instructions and hints. Here we also check not so much technical knowledge as the responsibility and learning ability of the candidate. Here in the photo below, someone is engaged in business, and someone smokes bamboo, because he was not prepared at home.

Basic technical knowledge is verified during a separate test. If the candidate does not understand what TCP / IP is, what a DNS server is for, what are the parameters of the household current, then it will be difficult for him to be an engineer on duty.

Points are counted immediately, the results are sorted out together. All points are entered into a “personal matter”, where they record all the results of subsequent exams and internships.

Introductory training. If the candidate has scored at least 21 points (70% of the possible), then he continues on his way and is invited to attend three-day introductory training.

On the first day of the future, the engineer on duty is introduced to the values ​​and rules of the company, the organizational structure and colleagues you need to know in person, talk about buns in the form of corporate events and holidays.

At the end of the first day - the first sightseeing tour of the data center, where they talk about the main functions of engineering systems.

The second day is completely devoted to the functional duties of the duty engineer. If before this was all in general terms, now the candidates analyze in detail the work with the monitoring system, the incident registration process, the regulations and instructions, the requirements for the processing time of requests for SLA.

Using specific examples, they are told what operations their area of ​​responsibility is limited to and what incidents need to be escalated to the second line of support.

The third day is completely devoted to the engineering infrastructure of the data center. The senior engineer of the site tells in detail about the main data center systems - power supply , cooling. Using the example of one of the sites, candidates are introduced to equipment and systems, and they are told about their role in the work of the data center. Future engineers will learn the principles of operation of industrial diesel power plants, UPS, air conditioners, chillers. On the same day, they disassemble the rack types and for the first time try to install and connect the equipment in the rack.

Also, the guys talk about the device network infrastructure.

Emergency situations are dealt with for each system and equipment. The main duty of the engineers on duty is monitoring, so they must understand how this or that problem will appear in the monitoring system and the main thing is what to do next.

At the end of a three-day intensive, candidates pass an exam on theory and practice. New material for these three days is a lot, and without practice it is difficult to assimilate everything to the end. We are well aware of this, and at this stage it is enough for us that the candidate show a clear understanding of his duties and basic knowledge about the engineering infrastructure of data centers.

All these three days the future leaders are looking at the guys. They evaluate their ability to work in a team.

Following the exam (you need to score at least 70% of the possible number of points), we decide whether to invite a candidate for an internship or not.

Internship. Paid internship under the training contract can last up to 3 months. The trainee helps the current shift on duty and continues his training on an individual plan. All this time a tutor has been working with him. Usually this is a senior engineer shift. He helps him to advance according to the plan of training. For each item, the future engineer on duty passes a mini-test, and at the end of the internship a great exam.

After the exam, the HR colleagues and the head of the support service again talk with the candidate to make sure that the candidate likes everything and still wants to work as an engineer on duty. The mentor gives feedback and shares his observations on the trainee's progress. If everyone is happy with each other, then the trainee is hired as an engineer on duty.

This is an interesting and rich way that a person who wants to become a duty engineer in our data center passes.

If you want to try your hand and get the opportunity to work in the largest commercial data center in Russia, send your resume to job@dtln.ru .