Expectations of development directions in the field of data storage and processing until 2020. Key Trends

The development of IT infrastructure is a steady process that was, is and will certainly be, gaining all just great momentum. Taking the next height of the standards for the implementation of their activities, new horizons are opening up for network engineers, the achievement of which becomes for them the next challenge and the basis of their daily labors. For the successful functioning of IT companies, it has always been very important to precisely define goals and directions for development, the most current trends, because as you can see, it is the IT industry that is at the cutting edge of technological progress and is particularly susceptible to innovation. In antiquity, for the opportunity to look into the day to come, people turned to oracles and artifacts, but now this so vacant activity is being hounded by professional associations, conducting all sorts of polls and creating on their basis the most versatile reports.

This very report , quite recently, pleased the IT community AFCOM (Association for Computer Operation Management). The subject of a full-scale study was the operation of data centers. According to the representatives of AFCOM, the presented report is a real event, as it covered the widest layers of specialists involved in the work of the data center (Data Center). Network engineers, managers, programmers and owners of providers companies in the presented report will give us a clearer vision of the situation in which the industry is now and where it will be in the next 3-4 years. Further, the article will present the main accents on which the above-mentioned report is based, which claims to be the most representative and the most relevant among all the similar ones conducted in 2015.
')

The future for mega data centers

Year after year, we hear about huge data centers, which are increasingly being erected by such leaders of the IT industry as Amazon, Facebook, Google, Yahoo, IBM, thus clearly generating a new trend. This situation will definitely continue in the coming years, which, in turn, will certainly lead to the dominance of these giant sites over smaller - familiar to us data centers. As a result of the onslaught of the giants, the total number of data centers will steadily decline, giving up flows of rapidly growing traffic volume from year to year to the largest infrastructure nodes.



“According to our calculations, by 2018, 72.6 percent of all hosting services provided will be placed in mega data centers, while the server area of ​​the total number of data centers mentioned will already be 44.6 percent. As of 2013, the area of ​​mega data centers from the total number of sites was only 19.3 percent, ”said Richard Villars, head of analytical department at IDC (International Data Corporation), on this occasion.

If IDC employees can take this accurately calculated percentage with a bit of skepticism, then the vector of data center integration itself does not cause any doubts among industry experts. Why progress has taken precisely this development path is not surprising, you can fully understand its origins based on the comments of representatives of two opposing sides: operators of mega data centers and network engineers operating small server platforms in enterprises and small companies.

“In most cases, the services provided by mega data centers will be cheaper and much better than their small counterparts,” said Adam Kramer, vice president of development for SUP SUPERNAP, uncompromisingly. “Our company made a bid not just for data giants, some of which are already successfully operating, our Fiber SuperLOOP project was really a breakthrough in understanding the existence of the IT infrastructure of the future. Having at our disposal a structure of a complex of mega data centers connected by fiber optics with a network delay not exceeding 7 milliseconds, we have a tremendous potential for instantly increasing capacity for our customers, redistributing peak load, and providing redundancy, which allows us to leave our competitors far behind. ” .

The position of representatives of large businesses is not difficult to understand, but what operators of not large server platforms have expressed on this issue came as a surprise. Most of the surveyed operators, although not in such a categorical form as Adam Kramer did, were unanimous in their prediction that over the next ten years most of the small server platforms in Europe and North America would cease to exist. Unlike the usual arguments for the flexibility of small data centers and the low cost of server rooms in enterprises, the emphasis was somewhat different.

“Nowadays, the company's network, isolated from the world, is rather an exception to the rules, which can be dictated by an exaggerated demand for its security. The need for employees to have access to the required resources of the enterprise network from anywhere in the world is already a given. Naturally, such a functional is within the power of providing every self-respecting company on its own, but, as practice shows, every year it becomes more and more expensive and meaningless. Increased demands on network loads, critical data availability 24/7, acceptable technical support response time - SLAs, clearly lead to an increase in bills for maintaining corporate networks, at the same time we see an ever-increasing variety of offers from hosting companies, the prices of which services from every year are becoming more affordable. Providing server-side fault tolerance at the TIER IV level for small and medium-sized companies is an impossible task at the same time, while for large data centers, the introduction of this standard has become quite commonplace, ”said Richard Willars.



According to the data presented by the Uptime Institute survey last year, only 18 percent of large companies decided to acquire their own server infrastructure, in the early years this percentage was much higher and reached 46 percent.

“Also an important factor in the mass departure from small server platforms, to large data centers in the near future, can be called a software environment. If you want to use a ready-made corporate software product, you will increasingly encounter the impossibility of using it at the “server back room” scale, this is due to the fact that even now the requirements of such products for network topology can be quite demanding. At a time when large companies can throw off ready-made software solutions and allow themselves to create their own working shells for online services, for small companies, in the case of which a small server could be quite effective, developing their own platforms is already a utopia. Overpaying for the use of hardware, cloud services can give the economy to the consumer by offering a software shell ready for use on a turnkey basis, ”noted Wikibon analyst David Floyer.

IT infrastructure is heading for landscaping

Since the creation of data centers, where the rack alone can consume up to 40 kW, is always a tangible event for local power grids, designing your own power generating capacity becomes a necessary step, but why does the choice increasingly fall on renewable energy sources? The question of the relevance of the use of renewable energy sources by data centers is rather ambiguous. IT equipment consumes with the same appetite the energy produced by both solar panels and the energy extracted from the burning of "non-ecological" hard coal, but experts are unanimous in their expectations - an ever closer interweaving of the IT industry and green technologies is coming. For such a perspective, there are two subjective, but no less important, factors: administrative, marketing. In the USA, Japan, countries of Europe there are very serious incentives, from subsidies to bans, companies using renewable energy sources, as well as non-CO2. From the point of view of marketing, in the modern world, not being “green” can sometimes be very risky for a business. The risk comes from both structured organizations and common sentiments of potential customers. Organizations like Green Peace, periodically creating "shameful lists" of companies producing pollution of the environment by greenhouse gases and nuclear waste, can easily undermine the functioning of the existing, or the creation of a projected data center.



“The managers of large companies-providers have no doubts about the need to use renewable energy sources, the problem with their implementation is only in the high cost of upgrading existing facilities,” says Matt Stansberry, content director at Uptime Institute “Apple is now considered the greenest companies , Facebook, Google, however, do not forget that the status of green does not make them the most efficient use of energy. To a certain extent, the public is misled. The data center on which the most efficient heat removal systems are used can be significantly more humane to the natural resources of the Earth than another data center, even if it is fully provided with renewable energy sources. ”

Unlike the employees of the Uptime Institute, representatives of provider companies who invest huge amounts of money in creating giant solar, wind and geothermal power plants with undisguised enthusiasm report on their success in “greening” their infrastructure, apparently this situation will only strengthen in the coming years.

"The place chosen by our designers to create Switch SUPERLOOP is truly unique, it allows us to effectively use not only solar, wind and geothermal energy, we also have the opportunity to use 93 percent of the technical water already used by other users in the cooling system" colleagues Adam Kramar “We have already signed a contract to provide our project with energy from a future solar power plant, the design capacity of which will be at least 100 MW. Our new station will be located near Las Vegas. ”

Optimization and planned cost reduction

The times of weak financial discipline of IT departments at enterprises and in scientific organizations have long gone into oblivion. Every year, the magic of flashing lights and diodes and the rhythmic noise of fans are less and less impressive in managing companies and, as a result, IT engineers are increasingly confronted with the practice of cutting down the budgets of their departments and face the struggle for improved performance.

According to a survey of organizations 451 Research, out of 1,200 respondents, only 17 percent said that their companies completely refused to support their own network infrastructure by transferring it to large data centers, eliminating the main problems associated with maintaining their own server capacity. At the same time, according to the aforementioned report, 49 percent of the respondents claimed that their companies actively use both the internal structure of data storage and processing and external, and this is already a great help for analyzing and searching for ways to optimize the network structure. As you can see from the above data, the “ghost” of increasing the efficiency of labor is hurt by quite a lot of IT specialists, both engineers and managers.



“When it comes to small data warehouses, our research shows very stable budgets allocated for these units. Despite the dramatically increasing needs and burdens on IT infrastructure, corporate executives do not want to significantly revise the item of expenditure on the IT component. Unlike small server-side engineers, their colleagues in large data centers meet the need to increase the efficiency of the IT infrastructure for several other reasons, the main reason is the existing tough competition in the services market, ”explains 451 Research Officer Tim Jefferson ( Tim Jefferson).

Touching this topic, a very characteristic parameter can be the power consumption of modern IT equipment. If in the 90s the average electricity consumption of the server rack was calculated at 1 kW per square meter, now this indicator starts from 6 kW and above, and this places new demands on the associated server components.

If the need for improvements in general is understandable (smaller, lighter, more economical), let's consider the main points of optimization that industry experts have highlighted in their surveys.

The fundamental rejection of the old decisions in favor of the new

For small data warehouses in enterprises, substantial optimization may consist in partial, or even complete abandonment of operating “internal” storages in favor of outsourcing this task to large data centers. As noted by the respondents, within the framework of the “server room” with dimensions of the order of tens of square meters, the return on small rational offers is often reduced to zero, which cannot be said about technical solutions on the scale of large data centers.

Dimensions

An accurate calculation of the needs of the IT infrastructure market at the design stage of the data center is a key component, the lack of capacity is just as detrimental for an efficient business as their excess.

Innovative facilities

The use of the most advanced materials and advanced technologies makes the functioning of server equipment much more efficient. The costs of cooling network equipment are becoming less and less every year, which is a result of the organization of cold and hot corridors in buildings of server racks, atmospheric cooling of racks and of course the use of heat sink by placing equipment in special liquids.

“Thanks to our work on upgrading the data center in California, the time to fully use the capacity of the cooling system was reduced to only 210 hours per year. At a level of 83 percent of the time, the upgraded data center and in general is ventilated solely by atmospheric air, which naturally undergoes pre-filtration. In addition, our specialists organized a system for distributing the load on equipment in such a way that customers with the most demanding projects were purposefully placed in areas of increased cooling, ”said Ron Pepin, executive director of NaviSite, and the main activity of the company is to optimize for existing data center.

The aforementioned words of Ron also confirm many of the survey participants who noted the tightening of various monitoring and as a result of control over, at times, the most insignificant parameters in the work of data centers.

Holistic vision

The construction of data centers not as outposts separated from the world, but as complementary parts of a puzzle, also contributed to optimizing the costs of their operation. Already, dozens of data centers are working, the excess heat created by which goes to the needs of the office, urban infrastructure, thereby returning part of the money spent to their owners.

Changes in the structure of everyday tasks of engineers serving the IT infrastructure

According to estimates of the people who received the survey, in the next 3-4 years, employees serving the corporate server infrastructure will feel particularly acute changes in their daily tasks. This tendency is actively manifested now and the reason for this is the increasing use of cloud services by companies. At a time when digital livelihoods of companies are increasingly migrating from server rooms to data centers, network engineers are gradually moving away from working with the “iron” part of the infrastructure and are increasingly joining with its “software” component.



“Dealing with the hybrid structure of our own server capacities and data centers, the problem of debugging and linking this whole economy into a single ecosystem becomes a serious challenge for many system administrators, I require new knowledge and experience from them” - Jeff Klaus - Intel Managing Manager. “Monitoring the temperature in the server room, reserving power supplies, laying communications, and other similar physical processes are no longer the main type of work of system administrators in enterprises. Since the organization of a software component, its monitoring and debugging often do not require the constant, physical presence of a specialist in an enterprise, this type of service will increasingly be outsourced. ”

As for the staff of large data centers, some of the respondents expressed the idea that some changes in everyday tasks will also hinder this level of employees in the IT sector. First of all, the changes will be affected not so much by technical specialists as by the managers working with clients. Faced with new realities in setting up data center customers with complex infrastructure based on the combination of their capacity and data centers, customer service managers will increasingly have to answer complex technical questions. As one of the interviewed specialists noted, “Soon we will have a situation when even the best sales people who do not have specific knowledge in the area of ​​interaction between software shells and networking among scattered local IT sites will seriously risk not fulfilling their quarterly sales plans.”

Frames decide everything

As for the personnel issue, the result of a survey of senior managers and owners of provider companies was quite interesting and somewhat unexpected. Almost all of the respondents showed that at these times the companies involved in the provision of data center services, are experiencing a serious personnel shortage, which by 2020 will only increase. What was surprising was not so much the fact of the lack of specialists, the whole IT industry suffers from this problem, the figures that 81 percent of managers face the problem of finding qualified personnel are unexpected, and the time to search for an employee that is required to close the current position is too long .



The main reasons for staffing hunger are actually several. First of all, this is of course the growth of production capacity, which requires more and more highly qualified specialists, the preparation of which takes more than one year. In the second place, the progress is being exaggerated, which gives rise to completely new vacancies for which specialists have to be pulled out from all sorts of related industries.

Mega data centers, such as those built by Switch SUPERNAP, are a very typical example of when hundreds of new vacancies are opened in a very short time in the face of existing problems with labor resources. Since by 2020 the growth of commissioning of giant data centers is expected, the problem with filling these IT nodes with network infrastructure becomes huge. As for the said company, its management sees the problem and takes it very seriously, establishing contacts with educational institutions in the state of Nevada.

“We work closely with the state educational system. Institutes, colleges and even schools are all the places where we select our future employees. Thanks to the planning of our activities, we can form a request for young professionals who, after graduation, will be able to find our first job in our company, ”noted Adam Kramer, Vice President of Switch SUPERNAP.

“Architects, environmentalists, heat and power engineers, water supply engineers and many others are those specialists with whom the data center operation had nothing to do and without which it is already impossible to imagine the work of a large data center today,” noted Karsten Sherer employee of the analytical company TEKSystem.

A very favorable situation for data center employees is emerging in terms of personal development prospects. The problem with personnel hunger, has become a real gift of fate for existing workers. As many surveyed ordinary employees point out, often their companies offer free refresher courses for them, the opportunity to get educational certificates at the expense of the company, and very attractive opportunities for rapid career growth, with an increase in the money allowance.

“Close to 72 percent of the providers of our provider companies that we surveyed said that they are planning or are already implementing targeted training programs for their new specialists, which are necessary for the successful operation of data centers of knowledge,” said Ted Lane, founder of Foote Partners research firm “Everything The managers interviewed by us also confirmed the logic that a specialist who received training and certification certificates legally have the right to claim an increase in their salaries. ”

Afterword

The above results of the aforementioned survey were conducted in the developed countries of Europe and North America, and for this particular region of the Earth they will be the most relevant. At the same time, they can be very interesting for employees and customers of hosting services in the post-Soviet space. The described trends are quite universal and relevant not only for the “Western” world but also for everything else, because they describe the problems of growth in the industry, which to some extent face on all continents. As can be seen from the above material, and in the next 3-4 years, the data storage and processing industry will be the most exposed: data center integration, greening, cost optimization, a certain shift in production tasks, and a shortage of qualified specialists. All these problems are relevant for us, except perhaps the critical use of renewable energy sources for data center robots, and this once again reminds us that we are firmly integrated into the modern world and are its integral component, sharing all its sorrows and joys, which dictates technological progress.