IT outsourcing: on the other side of the barricades

Topic inspired by reasoning xldsakamrhahn System administrator or outsourcing? about the quality of service by outsourcing IT companies. This article would like to show how IT outsourcing for small and medium-sized enterprises is a heavy and difficult-to-scale business. Today, technologies have become much more complex and capricious than before, and every year the situation becomes even more complicated, old technologies are being replaced with more and more new ones, and the knowledge and experience of a specialist without their constant updating and tracking trends quickly lose their relevance and become unnecessary. It is not for you to repair TVs, in the IT field devices are orders of magnitude more sophisticated and sophisticated.

Costs for most customers for IT services are equal to costs, for example, for utilities, i.e. the cheaper it costs - the better, the more profit and no secret that a small business like any other counts every penny. However, you should not put the price of services the main criterion for making decisions about cooperation with an outsourcing company and after reading this opus until the end you will understand why. Also at the end of the article, an IT company of a small size (up to 10 people) is waiting for a tasty bonus.

But let's take a look on the other side of the barricades; for the service company, the opposite is true - the more the customer pays, the more profit.

In fact, by providing services, outsourcing companies sell their time / their employees. Calculate, in the working week, 8 hours * 5 days of the working week = 40 hours of work of the system administrator at 100% load, but the employee will hardly work all 8 hours a day without regaining consciousness. The likely return in the region of 50-60% (if successfully adjust the workflow), i.e. subtracting the arrival to work is not exactly at 9:00, coffee, smoke breaks and transfers will be about 20-25 effective hours of work per week. Multiply, say, by 4 administrators, and now you have at your disposal fixed working capacities not exceeding about 100 man-hours per week. With, say, 20 clients will be about 5 hours per week for each client. Lot? Few? Keep in mind that this time also includes time for departures and for the delivery of equipment, the commissioning of a new client's workplace, maintenance of server equipment and work with new clients.
')
And here's the task, how to make the ratio of time spent / service quality / cost optimal? How to parallelize processes?

There are such approaches:

1. Most outsourcing companies operating in this market prefer to limit the number of trips and take money for additional trips above the limit. Everything is great, how to increase the profit of the outsourcing company? The first method is to make more additional visits, more visits over the limit of the tariff plan - more money. In this case, it is beneficial not to cure the problem, but to provide a service for patching holes. More admins - more clients can be served in this way, more profit. The disadvantages are obvious, there is no reason for administrators to make qualitative changes to the infrastructure, it is more profitable to just come and treat another incident once again.

2. But the second approach so far is our know-how, and among all the companies in Kiev, so far only we offer this approach initially. We thought, is it necessary to increase the number of visits to increase profit? And if you try to increase the cost of working hours of a specialist? Then you can hire better staff and provide better services. How to make it beautiful and unobtrusive? It turned out that with an unlimited tariff plan at about the same tariffs as in the first case, it makes sense to adjust the work as best as possible, so that at the same fixed rate for the client, the number of calls / departures should be kept to a minimum. And then with smaller quantities of departures and the same fixed rate, the cost of an hour of work automatically grows! And this is exactly what the client needs - fewer problems and what we need - a higher rate of employee work hours while maintaining the high quality of work. Conversely, when a large number of incidents occur, the cost per hour of a specialist’s work automatically drops. Looking ahead, I will say that in practice this approach works with many reservations for a spherical client in a vacuum.

That is why a poor-quality employee in our country not only does not make a profit, but also indirectly creates losses, since it requires control and “cleaning up” him, creating additional time costs.

3. Net hatching, taking into account the difference in rates for the qualifications of the contractor, required to solve the tasks. Probably, this is one of the most correct approaches, however, there are significant difficulties at the start with the presence of the necessary specialists, the criteria for assessing qualifications and determining the cost of costs. As an exercise, you can try to estimate the cost per hour of work of an Exchange specialist, Sharepoint, and enikechik.

Functional separation.

Unix? Yeah. We can do it, but better not.

Case number one.
A year ago, speaking on the phone with the director of a competitor, I heard the phrase “not without pride, but“ we are starting to deploy servers on clients on Linux ”.

Case number two.
Six months ago, a potential client called us with an unusual problem: “we have such cybernetics here, we cannot leave our incoming system administrator for service, because no one except him understands these technologies and we are afraid that we will not be able to work when he He will break everything remotely, he began to interfere in the affairs of the company and this is not good. ” The client subsequently did not dare to cooperate with serious alteration of the infrastructure.
But according to the results of our audit, it turned out that the two Linux servers worked as a cluster running virtual machines for xen and the company's web server and a domain controller for samba and VPN with a remote branch and other amazing things.

Yes, we also have freebsd servers for colocation and other interesting technologies on * nix systems, and the same asterisk telephony, but the client is trying to implement reinforced concrete, life-proven, simple as nail and clean, like a tear of a virgin, solution. Instead of a router for linux, we prefer the installation of linksys wrt54gl, or dlink dir-300 (400), not because we are not able to configure Linux, but because the client is no longer able to live with it. And even if the client chooses to go to the service for any other company - experts can even maintain life in windows servers and boxes-routers even with a not very high skill.

The introduction of new technologies is a useful and necessary thing, but it is preferable to think 10 times, is it worth it? If we are implementing, for example, a server under citrix, then it is necessary to think over, and whether there are at least two specialists with decent qualifications who can replace each other in the event of vacation, or, god forbid, injuries? Should I take on the additional burden of accompanying the decision if there is no clear answer to the question “what if it does not go as planned?” And for sure the introduction of additional technology will entail our additional time costs for solving problems.

We also save money to the client on the assessment of feasibility, for example, if there are no strict requirements for the availability of a mail server within the company, we do not implement the mailer inside, do not install an additional server, do not charge an additional fee for its maintenance and periodic adjustment of the anti-spam filter, and just deploy mail on google hosted services and everyone is happy.

Clone your brain.

I am deeply convinced that a person who knows the pitfalls of the profession and grew out of the pants of a self-organized, disciplined and responsible system administrator should run the outsourcing company. For me, there was a real revelation when the purchase of a rival company was considered that decided to close the business. Surprisingly, but the fact is that the company was managed by a person who has no relation to IT! The model of work is the departure of a specialist from morning to evening with clients, you can say "like a mouse in a circle." Perhaps we would be interested in buying a customer base, but it consisted mainly of clients such as government organizations located outside the city. There is a separate conversation about state companies, people do not work there, so we are going to avoid cooperation with such clients.

Handling calls and calls.

Obviously, the best thing to do when calling us is not to get on the robot with a voice greeting (everyone loves human communication) and not on the office manager, who will only record the call, in most cases without going into details and not specifying, but immediately on a competent specialist. who, if possible, will remotely try to give a qualified answer, either remotely solve the problem or delegate its solution with the necessary clarifications. But what if the specialist on duty went to leave, or went for a smoke break, because often their workload and employment is high? In this case, digital telephony comes to the rescue, with which we:
1. If there is no on-duty admin on the site, the call will be transferred along the chain to the mobile numbers.
2. Each call is recorded in the log automatically with the number of the caller, and if suddenly the call was not processed, we call the customer back with a clarification.
3. In the future, there will be a revision of the internal web-based accounting system with asterisk integration like CRM (there are difficulties with the cost of the GSM-SIP gateway and time to refine the system).

Pro business processes customers.

Not all companies understand their mission and in general what it is. In our country, it sounds like this: removing the headache of a client in the field of IT with reasonable savings. IT in most cases has the function of ensuring business continuity of production processes. Therefore, a deep understanding of the business processes taking place in the client’s company ensures success in providing services. Knowledge of what is critical for production, and what is secondary allows you to concentrate on what is really important and not to waste time on the “search for grail”. We do not have the right to introduce untested solutions, we do not have the right to play on the customer’s equipment with a self-learning bias, if this jeopardizes the reliability of business processes. Someone important is the application for working with customs declarations and mail, and some of the clients do not use mail at all, and most importantly - stable work of 1C. So simple is the secret of success that is revealed as the door to each client, an individual approach to each client. This does not always and everywhere work, very often in small companies business processes have not yet been formalized, in other words, it happens that “they themselves do not know what they want” and the organizational structure (hierarchy) is quite vague, for us this translates into that fact that we do not know with whom to resolve financial or organizational issues, so we decided to simply add a clause to the contract, which specifies the contact persons and their area of ​​responsibility.

It is necessary to clearly understand that the system administrator not only provides services for the treatment of computers, but also builds interpersonal relations with each of the client’s employees, therefore the appearance of a new administrator is accompanied by varying degrees of difficulty.
A typical company with 15 jobs usually has 15 employees, which means that the specialist will encounter 15 different characters, behavioral patterns and possible manifestations of psychological disorders of the client’s employees. Add to this the knowledge of the nuances of the operation of the equipment (someone has mailboxes of more than 7GB, yes, yes) and it becomes clear that the transfer of knowledge from one administrator to another becomes altogether difficult. Therefore, “in the subject line” there should be a cluster of at least two specialists who know the whole situation of the client, so that, in the case of force majeure, the new fighter could pick up the banner that had fallen from the hands of his colleague and carry it on with pride.

Knowledge Exchange.

The introduction of a knowledge sharing system is not only desirable, but also essential for work. The mechanism of the wiki is just what is needed in our case. However, the wiki also requires control over the content and update (someone has to draw the client’s network diagrams?). The principle of documenting everything and everything should be the main work, otherwise the knowledge will go along with your employee.

Solving problems that arise for users is also worthy of a separate verse in our song about service.

The following communication chains are possible to solve problems:
1. client's employee - admin. The perfect chain. Other interaction options will add a “tainted phone” effect.
2. client's employee - client's secretary - admin.
3. client employee - client secretary - lead manager of the outsourcing company - admin
4. Because we use a samopisnaya system of registration of applications in this link, you can add a ticket on the website.
client's employee - website - admin with their variations.
5. and so on

You can independently calculate the number of interaction combinations using combinatorial skills based on the data: client employee, client secretary, client guide, admin, website, admin guide. Slyusch, how slept, wah?

Communication problems.

One administrator is able to speak well, another is good to listen, the third is good at diagnosing and solving IT problems, the fourth writes beautifully reports. How many people can be found with all these qualities for an adequate salary? Experience shows that the task of a series of searching for needles in a haystack, which once again gives reason to think about keeping the found specialists (keywords: meals, mobile phones, transport, the general microclimate in the office, salaries).
Do you want to tell what causes mistakes in choosing an employee? Flared and rude, for example, or fancied himself a god, or the problem did not close in time, the necessary stress. An employee of the client goes to complain to his management, the management sends the complaint to me, I give the pill to the admin, in case of a relapse, parting follows.
What is it all about? All this leads to the loss of the client, and as a result, the source of income for the maintenance of the same admin.

Fixation of work performed. SLA.

Axiom: every client call must be recorded. All those who are faced with the need to report on the services provided understand the value of this axiom. It is also extremely useful to fix the time spent on solving a problem in reports for internal use and optimization of business processes. How much time is spent on closing the ticket, how much time is spent on the client, how many tickets did the client create and analyze the causal relationships, what is the effectiveness of each system administrator? How much to give bonuses and based on what criteria? All these questions must be sought and given reasonable answers.

Typical problems in working with the client.

1. Not all users are able to clearly and clearly explain the problem by telephone or other means, which makes operational diagnostics difficult.
2. The problem that has arisen with the employee of the client is initially positioned as VIN SISADMINA.
3. Viruses. Here there is an eternal war of good and evil, and the problem is as old as the world, in any case, first a new evil A1 / H1N1 appears, and only then antiviruses learn to catch and neutralize. Measures to be implemented are the closing of port 25 for all hosts except the mail and rolling up windows updates by installing an office WSUS server, which, however, should be closely monitored and selectively approved updates (additional time costs), because everyone knows the situation with licensing products, when with the next update the product is no, no, and it will start asking for money for activation (vendors, forgive me, how the policy of the country will change - we'll buy everything).

PROFIT.

To date, the company's regular expenses account for 70% of staff salaries, 20% of office and utility expenses. Once I decided to create a company in which I would like to work myself, and therefore we have free lunches for employees, albeit small but white salaries, payment of mobile communication expenses, summer-time motor transport (but often breaks, infection) . As the client base grows, the ratio of office expenses to income falls faster than salary costs.
In profit there are about 25% of irregular incomes - a one-time client, network installation, design work, purchase of equipment for the order.

Growth and scalability issues.

That brings an average customer income in the amount of 2500-3000 UAH (about $ 300-350), however you can come to this figure yourself, considering the cost of service on our website on the tariff calculator (most companies make the secret of their prices). 10 clients will bring about $ 3000- $ 3500 per month, of which $ 500 for salaries of 3-4 administrators will be about $ 1500-2000. Add the cost of meals, office, cell phones, and so on, and it will become clear that the tariffs on our website are currently understated. Because in most cases, the client is primarily interested in the issue price, frequent cold calls with questions from the series “how much will it cost to service xx computers and yy servers?” And not a word about HOW it will be serviced. Based on the above, each employee’s value is understandable for the company, as well as how difficult it is to find and raise the right one, and as a result, if 4 new customers come to us in one week, we will not be able to physically accept all of them for service! So it turns out that a large customer comes to us - we are not ready to serve because of a lack of hands, we cannot actively expand our staff because of a small profit, and that is where the new customers come in smoothly and slowly, while honing the edge of our business. processes.

And this is completely different from “bought 5 boxes of fish, sold, profit”.

I also fully understand the fullness of the market with students who “have enough beer for beer and okay” with the price of service two times lower than ours and with the corresponding quality, but at least 3 customers came to us after “having tasted such a service”.

Are there any fakapy?

Happen, but, fortunately, it is rather a rarity than a rule. Here is almost the only case that I don’t even want to remember, which took place just recently, due to the negligence of one of our employees, the chief accountant of the client lost part of the work in 1C base, because the base was in a ball on the server, and the virus from the Kido series dropped the network browser service on the server with an enviable regularity, i.e. access to the ball fell off, the DBF database was not recorded. The problem was not diagnosed by an employee in a timely manner (there was not enough qualification to understand why all of the services on the server are falling for no reason at all) and was not closed for a long time. Such an incident led to the involvement of an outside IT specialist who, with the help of two patches, became a national hero. Our image was trampled below the plinth and led to a rupture of relations with the client with the return of funds from our side for the entire period of cooperation (4 months), plus “penalties”. Well, swallowed, made the right conclusions, and, picking up snot, working on.

And finally, comrades, do not forget about criminal liability for installing “unlicensed software”, which may well be the reason for closing a business. And, if you carefully read the license agreement, you must “knock where it should be” when it detects the use of unlicensed.

Summing up, let me point out the idea from which the story began, how difficult and weakly scaled the business is with our organization of the process. Often you want to raise your right hand up and sharply lower with the words "well, it's all on ... this is IT," but, fortunately, it passes. After all, doing things you love, is not it?

Because, ladies and gentlemen, if not us, then who?

Promised bonus: Microsoft has a special Microsoft Active Pack Subscription licensing program for small IT companies , the issue price is $ 300 per year, and for this you get licensing for your organization 10 copies of windows (including 7k) and licensing of each server product released by Microsoft in one copy valid for 1 year plus the receipt of automatic licensing per unit of each new server product. Licensing can be renewed at $ 300 for each year.

Good luck to you and your customers, and may the force be with you.

Sincerely, Maxim Prokopov
IT Premium Company,
Kiev, Ukraine

References:
our website
RTFM book of a successful outsourcer (Moscow)
Microsoft Active Pack Subscription

Used Slang:
skill - English skill
profit - from English profit
fakap - with eng. fuck up
comrade - from English comrade, comrade