Virtualization and modernization in O & G: the story of one project
The dependence of enterprises and organizations on modern IT infrastructure is constantly growing, and with it the requirements for software and hardware complexes and systems and platforms for business applications that support them are growing. The IT infrastructure must be as reliable, productive and flexible as possible, ensure high speed of communications, support for various applications and services that allow solving tasks of the required level of complexity.
Virtualization projects based on x86 servers are not new to the Russian market. They are sold in large quantities in different industries. Today virtualization is the most frequently used technology in IT. No modern company can do without virtualization. It allows you to consolidate the computing resources of multiple servers and divide them among multiple applications according to the needs of these applications, that is, to use computational resources efficiently.
Virtualization provides the following benefits:
And this is not a complete list.
')
But how to successfully cope with this, at first glance, not very difficult and replicable task, how to improve the reliability and performance of many IT systems using virtualization?
This requires the ability to carefully design, competently implement solutions, listen and hear the opinion of the customer’s specialists. Then the virtualization system can be started in the shortest possible time and with the expected result. And for the customer the result is a significant increase in the efficiency of the units. One of such projects for the customer from the Russian oil and gas industry will be discussed.
First of all, it is worth noting the basic competence of the company implementing the project - the system integrator: along with the ability to competently build the infrastructure based on standard architecture servers and popular virtualization systems, knowledge and experience in the field of building virtualized server farms, its experts deeply understand the specifics of customer applications.
And although this project was fairly typical and not “tied” to the specifics of the application software, this knowledge was useful in the work both at the design stage of the solution, at the stage of migration and software launch, because the customer used a variety of geophysical applications and various IT infrastructure components. All these applications needed to be consolidated within a single virtualization system.
Typically, a typical virtualization implementation project begins with a study of applications that are planned to be migrated to a virtual environment, an audit of existing equipment to determine its actual wear and capabilities, or the selection of new IT platforms (this project just assumed the replacement of servers and the deployment of new disk arrays) . Then follows the development of the solution architecture and implementation specification.
After discussing all aspects of the project and approving the terms of reference, the virtualization and testing of the solution is implemented directly. At the final stage of the project, the effect of the implementation is assessed and the technical specialists of the customer are trained to administer the new infrastructure, if required. The customer also receives all the necessary technical documentation.
This project involved solving IT unification, consolidation and server virtualization tasks, but the main goals were:
The available disk capacity of the servers is already for the applications to work, and for deploying new applications additional computational capabilities were needed. This required upgrading the IT infrastructure and replacing outdated equipment.
Thanks to virtualization and hardware upgrades, we have been able to improve the reliability and performance of systems. Factors increasing the reliability of steel:
Performance has grown, primarily due to the greater processing power of the processors, speed and capacity of the RAM of the servers, as well as due to the introduction of a modern fast disk array as storage systems instead of local server disks. Microsoft Hyper-V was used as the virtualization hypervisor.
The list of priority servers for the application for migration includes eight servers. The implementation of the project also required the modernization of the storage system, a network infrastructure covering the two buildings.
In one of the buildings of the customer organization, a new disk array was installed. To configure replication, a reorganization of the addressing scheme in the network was required: taking into account the continuity of the customer’s production activities, the task turned out to be quite difficult. And it was not the only problem.
To migrate applications to new server platforms, physical machines were converted to virtual (P2V), but problems arose here, since the Microsoft Hyper-V virtualization platform selected by the customer does not include regular tools for converting Linux-based physical machines to virtual ones.
This can be realized only by converting these physical machines into virtual ones on the VMware ESXi platform and then converting the ESXi virtual machines into Hyper-V virtual machines, but the customer did not have a deployed virtualization environment on the VMware ESXi platform and physical servers on which such an environment could be to deploy.
In addition, most of the applications used by the customer on the Microsoft Windows platform work on computers with outdated versions of the operating system that are not supported by the current version of Microsoft Hyper-V. Therefore, it was impossible for them to convert physical machines into virtual ones using standard tools.
As a result, it was decided to create “empty” virtual machines under Linux and Windows in the Microsoft Hyper-V environment in order to reinstall the applications on them.
The customer uses many applications for processing and interpreting seismic data, for hydrodynamic modeling, for maintaining various databases. Among them are licensed applications of well-known vendors in the industry and proprietary software.
In particular, software systems are used to analyze field development indicators (BASPRO), to build geological models (BASPRO and IRAP RMS from ROXAR), to build hydrodynamic models (Tempest MORE from ROXAR). For processing and interpreting geophysical data, the software systems ProMAX, Focus, Geocluster, Paradigm, Landmark, Pangea, and Geomage MultiFocusing technology are used.
Such tasks are demanding of computing resources, bandwidth and latency in the network, bandwidth of the storage system, etc., and virtualization, as is known, reduces performance, but for relatively simple and small tasks it is also possible to use virtualized server farms. The customer of this project has truly large-scale tasks that a virtualized server farm cannot cope with in a reasonable time until they have emerged.
Initially, it was planned that a separate Hyper-V cluster would be deployed in each building, but it turned out that the outdated NetApp disk array installed in one of them could not be used for the Hyper-V cluster to work. it took the completion of the existing NetApp array (this work has not been completed yet), and some components of the supplied servers (SD cards) cannot be used to run Microsoft Hyper-V. In addition, it turned out that the project was weakly linked to the existing IT infrastructure of the customer - a reorganization of the data transmission network was required.
In practice, it turned out that many of the wishes of the customer in the framework of the tender specifications for the existing IT infrastructure cannot be implemented - a joint search for compromises was required. More careful design and better project management would avoid many of the problems encountered during its implementation (such as the inability to transfer physical machines from Linux to a virtual environment).
Despite the difficulties, the project was implemented by a team of system integrator and customer specialists in a short time. Here are its main steps:
The virtualization system now covers two buildings of the organization, located half a kilometer from each other - the administrative and production buildings. These buildings are connected by a fiber optic network. A part of the equipment supplied under the project is installed in the administrative building, the other - in the production building.
At the same time, it is possible to transfer virtual machines between buildings if necessary. The computational power and capacity of the new solution is sufficient for the operation of existing and the deployment of new applications.
We intentionally do not name the participants of this project. The purpose of the publication is to summarize the experience of such decisions. We hope that you will share your views with us, tell us about the successes and mistakes in the implementation of virtualization projects.
Virtualization projects based on x86 servers are not new to the Russian market. They are sold in large quantities in different industries. Today virtualization is the most frequently used technology in IT. No modern company can do without virtualization. It allows you to consolidate the computing resources of multiple servers and divide them among multiple applications according to the needs of these applications, that is, to use computational resources efficiently.
Virtualization provides the following benefits:
| Reducing equipment costs. | Achieved through the effective use of its resources. |
| Reduced power and cooling costs. | Provided by reducing the amount of equipment. |
| Increase staff performance | System administrators will be able to spend more time on performing truly important strategic tasks, rather than spending it on routine maintenance procedures. |
| Increased flexibility and availability of IT infrastructure | Virtual machines and applications can be migrated to other physical equipment without interruption of service. Reduced downtime. |
And this is not a complete list.
')
Problem to solve
But how to successfully cope with this, at first glance, not very difficult and replicable task, how to improve the reliability and performance of many IT systems using virtualization?
This requires the ability to carefully design, competently implement solutions, listen and hear the opinion of the customer’s specialists. Then the virtualization system can be started in the shortest possible time and with the expected result. And for the customer the result is a significant increase in the efficiency of the units. One of such projects for the customer from the Russian oil and gas industry will be discussed.
First of all, it is worth noting the basic competence of the company implementing the project - the system integrator: along with the ability to competently build the infrastructure based on standard architecture servers and popular virtualization systems, knowledge and experience in the field of building virtualized server farms, its experts deeply understand the specifics of customer applications.
And although this project was fairly typical and not “tied” to the specifics of the application software, this knowledge was useful in the work both at the design stage of the solution, at the stage of migration and software launch, because the customer used a variety of geophysical applications and various IT infrastructure components. All these applications needed to be consolidated within a single virtualization system.
Typically, a typical virtualization implementation project begins with a study of applications that are planned to be migrated to a virtual environment, an audit of existing equipment to determine its actual wear and capabilities, or the selection of new IT platforms (this project just assumed the replacement of servers and the deployment of new disk arrays) . Then follows the development of the solution architecture and implementation specification.
After discussing all aspects of the project and approving the terms of reference, the virtualization and testing of the solution is implemented directly. At the final stage of the project, the effect of the implementation is assessed and the technical specialists of the customer are trained to administer the new infrastructure, if required. The customer also receives all the necessary technical documentation.
Reliability and performance are in focus
This project involved solving IT unification, consolidation and server virtualization tasks, but the main goals were:
- Increased reliability through the decommissioning of obsolete servers and consolidation of applications on new platforms.
- Improved manageability through the introduction of virtualization.
- Standardization of the server park of the organization.
The available disk capacity of the servers is already for the applications to work, and for deploying new applications additional computational capabilities were needed. This required upgrading the IT infrastructure and replacing outdated equipment.
Thanks to virtualization and hardware upgrades, we have been able to improve the reliability and performance of systems. Factors increasing the reliability of steel:
- Server clustering, which allows virtual machines to be moved from one host to another in the event of a disaster or planned shutdown.
- Using instead of outdated platforms more modern servers, provided warranty support.
- Application in the IT infrastructure of a modern disk array with duplicated controllers instead of local disks of outdated servers.
Performance has grown, primarily due to the greater processing power of the processors, speed and capacity of the RAM of the servers, as well as due to the introduction of a modern fast disk array as storage systems instead of local server disks. Microsoft Hyper-V was used as the virtualization hypervisor.
The list of priority servers for the application for migration includes eight servers. The implementation of the project also required the modernization of the storage system, a network infrastructure covering the two buildings.
In one of the buildings of the customer organization, a new disk array was installed. To configure replication, a reorganization of the addressing scheme in the network was required: taking into account the continuity of the customer’s production activities, the task turned out to be quite difficult. And it was not the only problem.
Migration issues
To migrate applications to new server platforms, physical machines were converted to virtual (P2V), but problems arose here, since the Microsoft Hyper-V virtualization platform selected by the customer does not include regular tools for converting Linux-based physical machines to virtual ones.
This can be realized only by converting these physical machines into virtual ones on the VMware ESXi platform and then converting the ESXi virtual machines into Hyper-V virtual machines, but the customer did not have a deployed virtualization environment on the VMware ESXi platform and physical servers on which such an environment could be to deploy.
In addition, most of the applications used by the customer on the Microsoft Windows platform work on computers with outdated versions of the operating system that are not supported by the current version of Microsoft Hyper-V. Therefore, it was impossible for them to convert physical machines into virtual ones using standard tools.
As a result, it was decided to create “empty” virtual machines under Linux and Windows in the Microsoft Hyper-V environment in order to reinstall the applications on them.
Geophysical applications
The customer uses many applications for processing and interpreting seismic data, for hydrodynamic modeling, for maintaining various databases. Among them are licensed applications of well-known vendors in the industry and proprietary software.
In particular, software systems are used to analyze field development indicators (BASPRO), to build geological models (BASPRO and IRAP RMS from ROXAR), to build hydrodynamic models (Tempest MORE from ROXAR). For processing and interpreting geophysical data, the software systems ProMAX, Focus, Geocluster, Paradigm, Landmark, Pangea, and Geomage MultiFocusing technology are used.
Such tasks are demanding of computing resources, bandwidth and latency in the network, bandwidth of the storage system, etc., and virtualization, as is known, reduces performance, but for relatively simple and small tasks it is also possible to use virtualized server farms. The customer of this project has truly large-scale tasks that a virtualized server farm cannot cope with in a reasonable time until they have emerged.
It was smooth on paper ...
Initially, it was planned that a separate Hyper-V cluster would be deployed in each building, but it turned out that the outdated NetApp disk array installed in one of them could not be used for the Hyper-V cluster to work. it took the completion of the existing NetApp array (this work has not been completed yet), and some components of the supplied servers (SD cards) cannot be used to run Microsoft Hyper-V. In addition, it turned out that the project was weakly linked to the existing IT infrastructure of the customer - a reorganization of the data transmission network was required.
In practice, it turned out that many of the wishes of the customer in the framework of the tender specifications for the existing IT infrastructure cannot be implemented - a joint search for compromises was required. More careful design and better project management would avoid many of the problems encountered during its implementation (such as the inability to transfer physical machines from Linux to a virtual environment).
Despite the difficulties, the project was implemented by a team of system integrator and customer specialists in a short time. Here are its main steps:
| Equipment supply | November-December 2016 |
| System design | December 2016 - January 2017 |
| Installation and commissioning, acceptance testing | January 2017 |
The virtualization system now covers two buildings of the organization, located half a kilometer from each other - the administrative and production buildings. These buildings are connected by a fiber optic network. A part of the equipment supplied under the project is installed in the administrative building, the other - in the production building.
At the same time, it is possible to transfer virtual machines between buildings if necessary. The computational power and capacity of the new solution is sufficient for the operation of existing and the deployment of new applications.
We intentionally do not name the participants of this project. The purpose of the publication is to summarize the experience of such decisions. We hope that you will share your views with us, tell us about the successes and mistakes in the implementation of virtualization projects.
Source: https://habr.com/ru/post/333670/
All Articles