Tech support all over the world: yes, yes, these guys

Not so long ago, an article about the technical support department was published in the corporate blog of Veeam, where, unfortunately, our remarkable division was described very dryly. I do not like this situation, and I decided to try to correct the situation. I am an engineer working in this very department, we are at the pinnacle of customer attacks and sort out the most complex infrastructures in search of truth before assigning the status of a bug to a request and disturbing R & D. The entire text below was written by an engineer, approved by his colleagues, edited by them, and even published by Veeam technical support engineer. Yes, and a recent note , was surprisingly well received by the masters.

Initially, the post was conceived as descriptive, because I just wanted to show that technical support can be not only a dull center with tough operators, but also a place where professionals work, and a technical support officer should never be considered the lowest level in the IT industry . But in the end, the story turned out to be somewhat PR and since there are no plans to publish it in the profile blog, but rather the opposite, I think this is its undoubted plus.

Therefore, let me introduce you to a completely different view, coming from the very gut of a large Russian software company, on how technical support can and should work, how humanitarians are friends with techies and why we regularly receive letters of thanks from customers that use the phrase “You guys rock ”And“ You are the best technical support I have ever seen ”

Fotochki

I must immediately warn you that the article will not show photos of our business center causing salivation, jobs similar to the cabin of the spacecraft, and views from the office of the main city sights. But nevertheless, we are not in the barn and are not sitting on beer boxes.
As it should be in any adequate and successful company that cares about its employees, we work in a good business center, sit in comfortable chairs, at comfortable tables under which there are excellent computers, and you can always drink a glass of tea in any kitchen. with milk, eat a corporate waffle and dream about the beautiful in the rest room, the color of which will be most appropriate for your state of mind at the moment. And by itself all this surrounds the unobtrusive atmosphere of branding, flavored with decorations from employees.
But in order not to turn the article into a wall of text, we will add some of our photos under the spoilers. If interested - look, and our business to save your traffic.
And by the way, our office is closer to the metro than the Yandex office. And then in the comments skipped information that we are in a very awkward place.

Why do we need companies

Probably the main reason why the technical support department is considered inside the company not to be a burden or some kind of black hole, but a promising direction of development, is such a banal thing as money.
We bring the company a good profit and it imposes on us responsibility for the quality of our services. And here you can exclaim - Well, it's obvious! Well, who will pay for bad tech support?
And we will answer you - look at the cost of calls to the support of one good hypervisor and read reviews about the results. Paid support for some iron vendors regularly sends people to search for answers on the forums. And there are many such examples.
')
Therefore, for ourselves we decided that if to be professionals, then to be them to the end. If you support the software for backup and recovery of servers, then be kind to understand how these servers work. Better yet, how the infrastructure works, these servers are surrounding.

If, after recovery, the domain controller does not rise in combat mode, be able to find the cause and explain to the client how to avoid it in the future and what was his mistake in the past.
If the client cannot configure the hardware provider of his SAN to remove the VSS snapshots, try to find the cause and not immediately send it to the hardware vendor.
If during the backup the Hyper-V cluster falls apart, do not say that the problem is in the curved Windows and the old hardware. And even if the trouble is for all 146% not in our area of ​​responsibility, then explain it and make sure that you are understood in the right way, and only then say that in order to solve this problem he needs to contact a third company.

Even if you are wasting 15 minutes of your time, the client will remember that you tried to help him, although you were not obliged to do this at all. And oh, how they love and appreciate it. Karma will grow like dandelions in the summer. And in the end it will result in the extension of the contract for technical support, the increase in company income and the growth of your salary, too.

Our main principle is that we work with people, not problems. When you drop all production and you can’t instantly restore it from backup, the last thing you want to hear at the other end of the line is total indifference. Even from the strongest professional.

Who we are

Traditionally, there should be big words about a unique young ambitious team and super specialists. But this is boring and not interesting to anyone, so we will start from the beginning - with the selection of candidates.

To solve the tasks set before us, such a unique thing as a system administrator with above-average skills and fluent English is required. We write in our vacancies. And even better if he knows a less popular language, for example, Czech, Turkish, Hungarian, Spanish, etc. Where to get such a miracle, science is not known.
Therefore, we have developed our own style of hiring people and their subsequent training. For us, the main criteria are the presence in a person of potential and his desire to develop. We need to see this in the candidate, and his task is to help us see it.

You study in the last year, worked as a senior assistant to a junior sysadmin, your spoken English is at the level Landan from the capital of Gritbritan, but you know the language quite tolerably and calmly can write another message on the English-language forum, reading the untranslated documentation for the new piece of hardware? Welcome to us! For the first month we will tighten your technical part to the level we need, then we will see how you will respond to simple written requests, then we will answer the calls from India, Singapore and other countries whose residents do not give a damn about your pronunciation and grammar quality. Your language barrier will not collapse, it will literally evaporate. And there and the trial period will come to an end, and welcome to the world of big customers and many hours of sessions with native speakers.
You are not a unique case. We have more than half of the techies with such a history, and they are now talking to the British with the British so that they don’t shut up.

On the other hand, if you diligently studied at the philological department for 5 years and now you know the Mandarin dialect, the adverbs of the prairies of Mexico and the names of all the 60 types of snow that Chukotka people distinguish, or you simply know any foreign language (really any) and want to get since this is income, but you don’t want to sit in a translation office or drive around the streets, we are also ready to accept you into our friendly team. Of course, you will have to have a minimal knowledge base in IT, firmly distinguish the processor from the system manager and know how to install Windows. We will also teach you our wisdom with terrible words, protect you from complicated technical cases for all three months of probation, letting you fully experience the beauty of explaining the elementary truths to the most dull customers. And if you really want to develop and learn something new, then in 3 months you will be on a par with any newcomer of the department or even above all of them.
And do not think that it is difficult and borders on the impossible. These guys are almost half of the total, and all top performers (those who handle most of the calls and applications) do not even have a hint of technical education. They just wanted to learn and learn.

You are no longer new to IT, but you are tired of being a developer, web technologies are sitting in the liver, and maybe you are embarrassed to call yourself a sysadmin and deal with replacing cartridges in a buy-sell office or just want to change your place of work with benefits? We will be glad to you too. And to teach you less, and you will begin to bring benefit earlier. But the requirements for you will be more serious, and the tasks more interesting. And we will not forget to pay you money either.

I think it is also worth explaining a little why we do not care so much what language you know. We sell our products all over the world, which means that our customers speak the most unimaginable languages ​​and dialects. It is quite possible, if you search, then there is a couple in Maya.

Officially, we provide support only in English, French and German, but what does it mean officially?
If support in any language is official, it is fixed in the contract, which means that at any time, day or night, we must provide an engineer who speaks this language. The simplest example is that we have no official support in Russian. Guess why? The correct answer is that there are no Russian-speaking engineers in the American and Australian offices. Therefore, only in our office in St. Petersburg informal support is provided in Greek, Dutch, Spanish, Italian, Arabic and three French dialects (hello to the former colonies). I will not even write about the set of languages ​​in other offices. It is clear that in the states it is the languages ​​of South America, and in the eastern region it is just awful what some engineers can talk to. Yeah, Sinhala and Tamil are no joke.
By the way, our salespeople work in 14 languages. Or more.

That turned out our team

As you can see in the photo, we sit in large and bright rooms.

And it would be wrong to think that this greatly hinders communication with customers by phone. First, you can speak in a calm tone to your phone, and not shout like a tomato seller from the nearest market. Secondly, a good headset on both ears perfectly saves from cases of increased hubbub around. And thirdly, no one around is making a fuss. There are a lot of calls, but there are not a few of us, so the calls are perfectly “smeared” around the cabinets, and we do not interfere with each other, even if we really want to.

Contrary to the popular belief that open space is a universal evil, this approach gives us one more non-obvious advantage compared to the classic Saport people’s seating in the isolated compartments. At any time, you can look around the room and ask for help from a more experienced engineer, or in the case of a specific problem, call a specialist for this particular area. Like everywhere else, some of us have a better understanding of networks, others of mailers and databases, and still others know all the features of Hyper-V. Geniuses are able to keep everything in their heads, we have not yet met. Although it may have been looking bad.
And what is the point for us to hide from each other, instead of helping and sharing experiences? We're not enemies of each other, but we are doing a common thing.

This is very important because we are not a center where the secretary will answer you first, then another, then a dubious melody, and maybe, if the stars come together, they will switch to the most junior specialist. No, the caller gets us a qualified engineer immediately. Calls take all without exception, and it should be brought to the end by the one who picked up the phone. Discounts are made only for beginners on probation. We believe that the engineer should learn this way, diving headlong into the unknown.
It may seem that in this way we substitute a client when a person with insufficient competence answers his request, but when this happens, in those 10 seconds that the engineer asks for clarification of information on request, he will receive references from his colleagues to the necessary documentation indicating paragraph and brief recommendations on the answer. And in very thin moments, the second engineer can sit down on the call, who will be transparent for the client to help the call taker.
And, of course, we have senior T2 specialists and semi-mythical T3, who deal with the disentanglement of the most cunning cases, when the first line lacks knowledge and experience. They are the only ones who cannot be reached directly.

Why are we interested

I think it is no secret to anyone that virtualization is now one of the most fashionable and sought-after technologies. Fashionable only Big Data. Unfortunately, our country is still very far behind in this industry, but this does not bother us at all.
The general interest in resolving all cases leads to the fact that we bring to the logical end 99% of the calls. The remaining percentage is accounted for customers who indicated incorrect contact details or persistently do not answer calls.

During the shift, you can easily help revive a medium-sized data center consisting of dozens of hosts. After lunch, explain how to configure the newest story, a review of which I recently saw on the Internet and who stubbornly refuses to interact with the hypervisor exactly as expected. Well, closer to the end of the working day, to make a fight on Hyper-V, to which traditionally there is no intelligible technical documentation, and together with the client you feel yourself as a sapper in his ten-year cluster, raised through SCVMM, in which one of the DAG Exchange servers crookedly recovered an awkward movement you will become the father of the fall production system of one of the largest corporations in Europe.

And all this happens at the first level of support. The bulk of calls comes from senior system administrators and heads of IT departments. Of course, there are sad exceptions, but most of the calls are communication with people who have all the necessary competence for a mutually enjoyable dialogue.

If you wish, you can become a narrow specialist and pick up only complex cases from your favorite area. Suddenly the meaning of your being is tape libraries and are you ready to advise colleagues without a break for lunch and sleep? Or maybe you can, by the first two octets of the ip of the workstation, promote the entire routing of the multilayer network and explain to the customer why it’s time for him to dismiss the body responsible for the operation, albeit a virtual network.

And how many scripts and test software for all occasions are written inside the department, will not be countable soon. This is already starting to be given by the state in the state, but we are not greedy and actively make friends not only with developers and testers, but also with techwriters, because our mental health and the overall moral attitude of the entire organization will depend on how competently they write documentation. Even such a trifle as templates for letters is missing from us and this is a principled position at the highest level. Nobody likes to send and receive template letters. And the most active "writers" are engaged in writing articles for a public KB, under the supervision of the same technical writers, for the Chukchi are not a writer, but commas, Shaitaname, complex.

As a result, when a person moves to another department or quits, which happens much less often, he leaves as a specialist of a very good sysadmin. Or a developer, or a team leader. At this point, who in what steppe wanted to go deep while he was under our caring wing?

And in general, we strongly for the manifestation of individuality and never limit the desire to develop in any area. If an engineer can explain how his interest in a particular area can benefit the team, then let him engage in self-development even during working hours, but of course, not to the detriment of the production process.

And at the end, the traditional

If habrasoobschestvuyu will be interested in stories from the life of backup copies, I think we can make a good selection of epic rescues, creepy files and just funny stories from the life of backup systems engineers. So what, and we have enough fun.

And of course, if you have questions about our department in particular, or about the organization as a whole, feel free to write their messages. We will try to answer all.


de finale