Oblakoteka has selected Lenovo servers as a platform for the latest IaaS services
In December 2016, the Oblakoteka company, a member of the MONT group of companies and specializing in the organization and deployment of private clouds and IT infrastructures of customers on its own platform and on the Windows Azure platform from Microsoft, announced the launch of the IaaS service “Virtual Infrastructure on Windows Server 2016.
It should be noted that since 2012, the company has deployed and is operating in commercial mode the IaaS platform based on Microsoft Hyper-V 2012 R2 - two clusters (one in each data center) with redundancy of computing nodes in the N + 2 mode.
The basis of the hardware component of the new IaaS service on the Windows Server 2016 platform is the Lenovo x3650 M5 servers and Mellanox network equipment.
')
Lenovo's high-performance server hardware was specifically tested and prepared for use with the Windows Server 2016 software platform. To ensure high performance, a platform was deployed in a hyper-convergent environment with a flash array based on the Storage Space Direct technology, which minimizes storage latency.
This is one of the first in Russia implementations of the IaaS service on the latest Microsoft Windows Server 2016 virtualization platform. The service is provided as part of the Microsoft COSN Russia initiative from data centers in Russia and provides unique opportunities for ensuring the security and performance of virtual resources. It is managed through the familiar Windows Azure Pack interface, familiar to Microsoft Azure users.
Hyper-converged platform: a cluster of servers for hosting virtual machines.
In particular, Shielded VMs technology provides reliable data protection of virtual machines, including from the provider's administrators. Container technology greatly simplifies software life cycle management for developers. Templates of virtual machines and Nano Server reduce costs, allow you to place many VM roles on a minimal amount of resources.
The new platform also has a built-in network load balancer, which makes it possible to dynamically change the capacity of a dedicated VM of RAM and the storage, the computational capacities involved by it, offers many other tools. Now it is a full-fledged service, which includes both the Windows Server 2016 virtualization platform and the Windows Server 2016 features on guest virtual machines.
The new energy efficient Lenovo System x3650 M5 servers in the 2U package are productive, scalable and compact platforms built on 8-core AMD and Intel processors, have 768 GB RAM and use solid-state SSD drives.
The features of this system are: hot-swappable drives and power supplies, various disk basket options, 9 PCIe 3.0 slots, and the ability to expand the functions of a RAID controller.
These servers are in demand in those areas where it is necessary to perform cloud applications, productive computing, delivery of media data and other tasks. Lenovo System x3650 M5 servers are designed for high workloads in 24/7 mode, often used for hosting services, used as a platform for virtualization, terminal servers, distributed and high-performance computing, database servers and applications.
Servers support centralized resource management, provide reliable data encryption and protection. Diagnostic tools allow you to quickly find and correct errors, and hot swapping of fans and drives helps, if necessary, to quickly repair. The network infrastructure is built on Mellanox ConnectX-4 Lx Dual Port network adapters and SN2100 switches (with 100GbE ports).
The ConnectX-4 Lx EN network controller with support for switching solutions for Ethernet with a capacity of 1/10/25/40/50 Gbit / s is designed to meet the challenges of virtualized infrastructures and has the highest performance in its class for various applications and markets. These adapters combine true hardware I / O isolation with scalability and efficiency, making them an economical solution for cloud infrastructures, databases, and storage platforms.
ConnectX-4 Lx EN supports speeds of 1, 10, 25, 40 and 50 GbE, provides a delay of less than 1 microsecond and a message transfer rate of 75 million packets per second. They contain hardware support for RDMA over Converged Ethernet, mechanisms for unloading Ethernet protocols without state control, overlay networks, and GPUDirect technology.
We asked the representatives of Oblakothek to tell about the data deployment and about the features of the project, which allowed us to offer the most modern technologies to our clients and partners.
“Oblakoteka” - so far the only Russian cloud provider offering IaaS services based on Windows Server 2016?
Yes it is. At the time of launching the service in Russia, there was not a single provider who built the IaaS service on the Windows Server 2016 platform. Some only supported this OS as a guest on virtual machines. According to our data, we are still the only such provider, more than six months already.
Why were Lenovo servers chosen as the virtualization platform, in which configuration? What means have you managed to achieve high service performance?
The idea of ​​creating a service arose in August 2016 together with the Russian representative office of Microsoft. The most important requirement was that the vendor had a hardware complex tested for compatibility with Windows Server 2016 in terms of deploying a Storage Space hyperconvergent cluster and support for Shielded VMs.
In addition, the equipment had to be delivered in a very short time of delivery, in fact - from a warehouse in Moscow. Lenovo met all these requirements, it already had experience in deploying such a solution on its platform and was ready to provide us with the necessary assistance in choosing and setting up the server configuration for this task.
Together with Microsoft, we identified the configuration requirements. As a storage subsystem, it was decided to use only flash drives and use NVMe drives for caching. This approach led to excellent performance in the performance of the virtual disk array.
The architecture of the solution was developed by Oblakothek specialists together with the Russian representative office of Microsoft. In working out the details of the project, Microsoft's recommendations and solutions from Lenovo were studied.
What is the role of network infrastructure? What was the reason for choosing equipment from Mellanox? What is the network bandwidth? What means is SDN implemented?
For any hyperconvergent solution, network infrastructure is a critical element. All management, data exchange, disaster recovery is carried out through the network and therefore the requirements for the quality of its work are very high. In the case of S2D ( Storage Spaces Direct ), SMB3 protocol support is required. It is most fully implemented in the equipment Mellanox.
The chosen solution allowed us to build a highly available data transmission network with a speed of up to 50 Gbit / s per node. Software-configured network ( Software-Defined Networking , SDN) is built using the Windows Server 2016 operating system: it provides for the roles of the SDN controller, balancer, and other related roles.
On the basis of high-performance hardware Lenovo and Mellanox running Windows Server 2016 implemented hyperconvergent environment with SDNv2. What exactly is platform hyperconvergence? What is the difference SDN "new generation"?
The hyper-converged architecture combines computing, network, and storage resources into one layer. Windows Server 2016 allows you to deploy Storage Management Services (SDS), Network (SDN), and Hypervisor (Hyper-V) in fault tolerant mode on the same hardware. Unlike the previous version of SDN, important services have been added to the new one, for example, a load balancer - LoadBalancer (SLB).
Hyper-convergent systems do not have a single point of failure, there are ample opportunities for horizontal scaling: if you need to increase the resources and performance of the cluster to operate additional VMs, as well as increase the capacity and performance of the storage, then simply add new nodes to the cluster. Instead of a SAN, a regular local area network (SDN with RDMA) is used. Therefore, there is no need to buy expensive FC, FCoE solutions or configure iSCSI. With the new features of SDS and SDN using Windows Server 2016, you can create hyperconvergent cloud solutions.
Tell us about the server storage subsystem with N2215 SAS / SATA HBA and storage pool, PCI NVMe, SATA SSD cards. How is it built, is it used?
Storage Spaces Direct (S2D) storage system is based on local storage with NVMe SSD for caching. For S2D, it is important to comply with hardware compatibility requirements. So you cannot use RAID controllers to connect pool disks (even in HBA mode). Therefore, only compatible HBAs are used.
Since in our pool there are only SSD drives, the tearing does not make sense. According to Microsoft recommendations, we use PCI drives with NVMe interface for caching. They significantly speed up write operations compared to conventional SSDs.
To ensure high performance and taking into account all the features of the new operating system, a platform has been deployed on a hyper-convergent environment with a flash array based on the Storage Space Direct technology. Tell us about this component of the project.
Today, SDS (Software Defined Storage) technology, software-configured storage, is becoming increasingly popular. Many vendors offer their own SDS solutions (VMWare VSAN, HPE / Nutanix, Ceph, etc.). In Windows Server 2016, Microsoft offered its SDS version - Storage Space Direct (S2D). It allows you to build a highly accessible, productive storage system on ordinary servers with local drives of various types.
In this case, you can use the technology of classic storage systems of high-end models - caching, tearing, high availability at the racks and data centers. In the case of hyperconvergent configuration, servers "see" storage as a classic Windows cluster disk.
If it is necessary to provide access to the storage for external customers, then you can use classical technologies - iSCSI, SMB, NFS. At the level of the virtual disk, you can control the IOPS parameters, determine the upper and lower bounds by the number of IOPS normalized to the desired size of the data block.
What features of Windows Server 2016 are already in demand or can they be demanded by customers? How do you assess the prospects of protecting confidential customer data from access by the service provider employees (Shielded VMs)? What new technologies are involved for faster deployment of virtual machines and applications?
Windows Server 2016 contains many innovations. The newest built-in server protection components in Microsoft Windows Server 2016 ensure the prevention of attacks and the detection of suspicious actions. The issue of data security and protection for a number of users is very important. And the threat is considered not only external, but also internal, from the side of the service provider. Now you can restrict access to the data inside the virtual machine to the provider's administrators. This will help this category of clients to decide on the transition to the cloud. Shielded VMs - the safest way to protect VM tenants from intruders with administrator rights.
In addition to Shielded VMs, our development users can use container technology. At facilities with graphic accelerators, you can now comfortably work with CAD programs, animation controls, etc. without the use of third-party additional software. For administrators, the Nano Server appeared - the safest runtime mode of Windows Server 2016, as well as the “built-in” Load Balancer balancer.
Tell us a little about the progress of this project. How was all this created, what were the difficulties, what did you manage to solve, and what was put into the tasks “for the future”?
The main problem is the paucity of documentation and diagnostic tools. Most of the time we spent on troubleshooting caused by seemingly minor errors in the configuration of the network adapter driver. We worked very closely with the Lenovo support team to fulfill Host Guardian Services’s requirements for the cluster servers (used by Shielded VMs). And to launch all the features SDN actively communicated with Microsoft experts.
Now, with each passing day, the amount of available information is growing, documentation is improving. If we took the project today, it would be easier.
Until June, the stand is in test mode. What are the future plans? Do you plan to migrate from Azure Pack to Azure Stack? What can be expected in this regard?
The task of the near future is to transfer the basic infrastructure to Windows 2016 and further develop the service. A full-fledged stand with a load close to real is an excellent opportunity to go through the stages of training employees and technology “childhood diseases” without harming commercial customers.
Despite the similarity of names, Azure Pack and Azure Stack are completely different platforms with fundamentally different approaches. The Azure Stack has significantly more services, but the provider’s control, customization, management and quality assurance is significantly less. In addition, the Azure Stack is a fundamentally new model of monetization and it has not yet been definitively defined. In general, we currently believe that they do not replace each other, but rather are designed for different target audiences.
How do you evaluate the results of the project?
We always try to offer the most modern technologies to our customers and partners as quickly as possible in order to give them competitive technological advantages. Hyper-converged solutions based on Windows Server 2016 and Lenovo server hardware are a simple and economical way to implement a cloud of any scale and deploy modern cloud services.
It should be noted that since 2012, the company has deployed and is operating in commercial mode the IaaS platform based on Microsoft Hyper-V 2012 R2 - two clusters (one in each data center) with redundancy of computing nodes in the N + 2 mode.
The basis of the hardware component of the new IaaS service on the Windows Server 2016 platform is the Lenovo x3650 M5 servers and Mellanox network equipment.
')
Lenovo's high-performance server hardware was specifically tested and prepared for use with the Windows Server 2016 software platform. To ensure high performance, a platform was deployed in a hyper-convergent environment with a flash array based on the Storage Space Direct technology, which minimizes storage latency.
This is one of the first in Russia implementations of the IaaS service on the latest Microsoft Windows Server 2016 virtualization platform. The service is provided as part of the Microsoft COSN Russia initiative from data centers in Russia and provides unique opportunities for ensuring the security and performance of virtual resources. It is managed through the familiar Windows Azure Pack interface, familiar to Microsoft Azure users.
Hyper-converged platform: a cluster of servers for hosting virtual machines.
In particular, Shielded VMs technology provides reliable data protection of virtual machines, including from the provider's administrators. Container technology greatly simplifies software life cycle management for developers. Templates of virtual machines and Nano Server reduce costs, allow you to place many VM roles on a minimal amount of resources.
The new platform also has a built-in network load balancer, which makes it possible to dynamically change the capacity of a dedicated VM of RAM and the storage, the computational capacities involved by it, offers many other tools. Now it is a full-fledged service, which includes both the Windows Server 2016 virtualization platform and the Windows Server 2016 features on guest virtual machines.
The new energy efficient Lenovo System x3650 M5 servers in the 2U package are productive, scalable and compact platforms built on 8-core AMD and Intel processors, have 768 GB RAM and use solid-state SSD drives.
The features of this system are: hot-swappable drives and power supplies, various disk basket options, 9 PCIe 3.0 slots, and the ability to expand the functions of a RAID controller.
These servers are in demand in those areas where it is necessary to perform cloud applications, productive computing, delivery of media data and other tasks. Lenovo System x3650 M5 servers are designed for high workloads in 24/7 mode, often used for hosting services, used as a platform for virtualization, terminal servers, distributed and high-performance computing, database servers and applications.
Servers support centralized resource management, provide reliable data encryption and protection. Diagnostic tools allow you to quickly find and correct errors, and hot swapping of fans and drives helps, if necessary, to quickly repair. The network infrastructure is built on Mellanox ConnectX-4 Lx Dual Port network adapters and SN2100 switches (with 100GbE ports).
The ConnectX-4 Lx EN network controller with support for switching solutions for Ethernet with a capacity of 1/10/25/40/50 Gbit / s is designed to meet the challenges of virtualized infrastructures and has the highest performance in its class for various applications and markets. These adapters combine true hardware I / O isolation with scalability and efficiency, making them an economical solution for cloud infrastructures, databases, and storage platforms.
ConnectX-4 Lx EN supports speeds of 1, 10, 25, 40 and 50 GbE, provides a delay of less than 1 microsecond and a message transfer rate of 75 million packets per second. They contain hardware support for RDMA over Converged Ethernet, mechanisms for unloading Ethernet protocols without state control, overlay networks, and GPUDirect technology.
We asked the representatives of Oblakothek to tell about the data deployment and about the features of the project, which allowed us to offer the most modern technologies to our clients and partners.
“Oblakoteka” - so far the only Russian cloud provider offering IaaS services based on Windows Server 2016?
Yes it is. At the time of launching the service in Russia, there was not a single provider who built the IaaS service on the Windows Server 2016 platform. Some only supported this OS as a guest on virtual machines. According to our data, we are still the only such provider, more than six months already.
Why were Lenovo servers chosen as the virtualization platform, in which configuration? What means have you managed to achieve high service performance?
The idea of ​​creating a service arose in August 2016 together with the Russian representative office of Microsoft. The most important requirement was that the vendor had a hardware complex tested for compatibility with Windows Server 2016 in terms of deploying a Storage Space hyperconvergent cluster and support for Shielded VMs.
In addition, the equipment had to be delivered in a very short time of delivery, in fact - from a warehouse in Moscow. Lenovo met all these requirements, it already had experience in deploying such a solution on its platform and was ready to provide us with the necessary assistance in choosing and setting up the server configuration for this task.
Together with Microsoft, we identified the configuration requirements. As a storage subsystem, it was decided to use only flash drives and use NVMe drives for caching. This approach led to excellent performance in the performance of the virtual disk array.
The architecture of the solution was developed by Oblakothek specialists together with the Russian representative office of Microsoft. In working out the details of the project, Microsoft's recommendations and solutions from Lenovo were studied.
What is the role of network infrastructure? What was the reason for choosing equipment from Mellanox? What is the network bandwidth? What means is SDN implemented?
For any hyperconvergent solution, network infrastructure is a critical element. All management, data exchange, disaster recovery is carried out through the network and therefore the requirements for the quality of its work are very high. In the case of S2D ( Storage Spaces Direct ), SMB3 protocol support is required. It is most fully implemented in the equipment Mellanox.
The chosen solution allowed us to build a highly available data transmission network with a speed of up to 50 Gbit / s per node. Software-configured network ( Software-Defined Networking , SDN) is built using the Windows Server 2016 operating system: it provides for the roles of the SDN controller, balancer, and other related roles.
On the basis of high-performance hardware Lenovo and Mellanox running Windows Server 2016 implemented hyperconvergent environment with SDNv2. What exactly is platform hyperconvergence? What is the difference SDN "new generation"?
The hyper-converged architecture combines computing, network, and storage resources into one layer. Windows Server 2016 allows you to deploy Storage Management Services (SDS), Network (SDN), and Hypervisor (Hyper-V) in fault tolerant mode on the same hardware. Unlike the previous version of SDN, important services have been added to the new one, for example, a load balancer - LoadBalancer (SLB).
Hyper-convergent systems do not have a single point of failure, there are ample opportunities for horizontal scaling: if you need to increase the resources and performance of the cluster to operate additional VMs, as well as increase the capacity and performance of the storage, then simply add new nodes to the cluster. Instead of a SAN, a regular local area network (SDN with RDMA) is used. Therefore, there is no need to buy expensive FC, FCoE solutions or configure iSCSI. With the new features of SDS and SDN using Windows Server 2016, you can create hyperconvergent cloud solutions.
Tell us about the server storage subsystem with N2215 SAS / SATA HBA and storage pool, PCI NVMe, SATA SSD cards. How is it built, is it used?
Storage Spaces Direct (S2D) storage system is based on local storage with NVMe SSD for caching. For S2D, it is important to comply with hardware compatibility requirements. So you cannot use RAID controllers to connect pool disks (even in HBA mode). Therefore, only compatible HBAs are used.
Since in our pool there are only SSD drives, the tearing does not make sense. According to Microsoft recommendations, we use PCI drives with NVMe interface for caching. They significantly speed up write operations compared to conventional SSDs.
To ensure high performance and taking into account all the features of the new operating system, a platform has been deployed on a hyper-convergent environment with a flash array based on the Storage Space Direct technology. Tell us about this component of the project.
Today, SDS (Software Defined Storage) technology, software-configured storage, is becoming increasingly popular. Many vendors offer their own SDS solutions (VMWare VSAN, HPE / Nutanix, Ceph, etc.). In Windows Server 2016, Microsoft offered its SDS version - Storage Space Direct (S2D). It allows you to build a highly accessible, productive storage system on ordinary servers with local drives of various types.
In this case, you can use the technology of classic storage systems of high-end models - caching, tearing, high availability at the racks and data centers. In the case of hyperconvergent configuration, servers "see" storage as a classic Windows cluster disk.
If it is necessary to provide access to the storage for external customers, then you can use classical technologies - iSCSI, SMB, NFS. At the level of the virtual disk, you can control the IOPS parameters, determine the upper and lower bounds by the number of IOPS normalized to the desired size of the data block.
What features of Windows Server 2016 are already in demand or can they be demanded by customers? How do you assess the prospects of protecting confidential customer data from access by the service provider employees (Shielded VMs)? What new technologies are involved for faster deployment of virtual machines and applications?
Windows Server 2016 contains many innovations. The newest built-in server protection components in Microsoft Windows Server 2016 ensure the prevention of attacks and the detection of suspicious actions. The issue of data security and protection for a number of users is very important. And the threat is considered not only external, but also internal, from the side of the service provider. Now you can restrict access to the data inside the virtual machine to the provider's administrators. This will help this category of clients to decide on the transition to the cloud. Shielded VMs - the safest way to protect VM tenants from intruders with administrator rights.
In addition to Shielded VMs, our development users can use container technology. At facilities with graphic accelerators, you can now comfortably work with CAD programs, animation controls, etc. without the use of third-party additional software. For administrators, the Nano Server appeared - the safest runtime mode of Windows Server 2016, as well as the “built-in” Load Balancer balancer.
Tell us a little about the progress of this project. How was all this created, what were the difficulties, what did you manage to solve, and what was put into the tasks “for the future”?
The main problem is the paucity of documentation and diagnostic tools. Most of the time we spent on troubleshooting caused by seemingly minor errors in the configuration of the network adapter driver. We worked very closely with the Lenovo support team to fulfill Host Guardian Services’s requirements for the cluster servers (used by Shielded VMs). And to launch all the features SDN actively communicated with Microsoft experts.
Now, with each passing day, the amount of available information is growing, documentation is improving. If we took the project today, it would be easier.
Until June, the stand is in test mode. What are the future plans? Do you plan to migrate from Azure Pack to Azure Stack? What can be expected in this regard?
The task of the near future is to transfer the basic infrastructure to Windows 2016 and further develop the service. A full-fledged stand with a load close to real is an excellent opportunity to go through the stages of training employees and technology “childhood diseases” without harming commercial customers.
Despite the similarity of names, Azure Pack and Azure Stack are completely different platforms with fundamentally different approaches. The Azure Stack has significantly more services, but the provider’s control, customization, management and quality assurance is significantly less. In addition, the Azure Stack is a fundamentally new model of monetization and it has not yet been definitively defined. In general, we currently believe that they do not replace each other, but rather are designed for different target audiences.
How do you evaluate the results of the project?
We always try to offer the most modern technologies to our customers and partners as quickly as possible in order to give them competitive technological advantages. Hyper-converged solutions based on Windows Server 2016 and Lenovo server hardware are a simple and economical way to implement a cloud of any scale and deploy modern cloud services.
Source: https://habr.com/ru/post/325558/
All Articles