“Cubes” for stores: why hyper-convergence is really needed, and why it is not just a buzzword

Old infrastructure

There are 8 large stores with an area of ​​more than 10 thousand squares each. Each store has an office with users and documents. At each point there is a server node - trading applications, file server, domain controller, other services. The communication channel is very thin, it is defined by a foreign corporate standard. It is enough exactly for administrative actions and synchronization of the base with the accumulated per day for the whole night. There is no talk of any synchronous or asynchronous replication of the database with the data center - only the night mode for sending diffs. Backup to a streamer. On the wall was a manual, according to which store employees changed cartridges once a day.

In such conditions, we implemented Simplity - one of the first projects to introduce solutions of this class in Russia. The request did not come in the form of “prompt solutions”, but in the form of a specific task “There is so much power, such volume is needed”. Then a set of five expensive pieces of iron was obtained, or of two expensive ones, but on an unfamiliar shaman's Simplity. Chose the second. It turned out a single infrastructure with a single space and such a slow exchange between sites. Very strange thing.
')
Now I will tell you what the devil-system does. Running a little ahead - there is a trendy hyperconvergence and the main feature - global deduplication.



Each store has 4-5 virtual machines. Some of them relate to infrastructure: domain controller, file server. Part of - specific applications for trade: weights management, printing of price tags and posters, analytics services. All these tasks fit into a small system of two SimpliVity OmniCube CN-1400 nodes.

As you can see, in this project, SimpliVity OmniCube are clusters of two nodes. VMware is placed on a cluster, in each server there are internal disks. Disk space is issued on a virtual machine. The useful space is combined into a single storage system - and sent back to the server. SDS as ScaleIO or RAIDIX , but the functionality is designed a bit for something else. The main thing is that you do not need to have a separate data storage system, only server disks.

The nodes are interconnected via a 10 GbE network directly, so there is no need for expensive 10 Gbps switches in stores. Virtualization Software - VMware vSphere. It turned out a typical set of infrastructure for the new store "in the box." When you open a new store, you don’t have to think again about the server hardware, it’s enough to take a couple of SimpliVity cubes. SimpliVity clusters in different stores see each other and are managed centrally. VM backups are made once a day and stored for two weeks. One backup is stored locally to quickly pick up data when it is accidentally deleted or logically corrupted. The second is stored on SimpliVity in a nearby store. The storage of the second cluster does not depend on the first.

This is quite expensive based on one implementation (can be made similarly cheaper on other solutions), but if you count along with the time of the administrator, cooling, energy, cost of support and its extension in the future, and take into account that all this stuff replaces the entire traditional stack / infrastructure, it turns out very well and nicely. The fact is that there is an “all in one”, a very simple set of components. You need all poladmina with knowledge of VMwary. The solution easily scales to any number of nodes. The main thing is that there is no mega-fat database, for such very large kids have all-flash arrays. The main case is branches and user terminal farms. The testing process also revealed that this thing is very nice working with bad channels - thanks to the built-in optimization of communication channels and built-in deduplication on the fly.

As a result, stores open several pieces a year in addition to the eight that already exist. This is the size of the Metro type. Offices in them for 30 active users. They chose a typical IT infrastructure solution for local applications. It was planned to open new stores, and in existing ones it was necessary to change obsolete hardware. Since IT professionals are not only involved in infrastructure, the solution should be as simple as possible and managed centrally. Ideally, the customer was looking for one ready-made box, which can be replicated in stores and generally forget that they have IT infrastructure.

The second requirement is to store backup copies of the store data on a remote site. It didn't make sense to transfer backups via the communication channel, as it worked for a long time. The data should be stored at different geo-points, and administrators understood this very well.

Testing

Here is a report from our test solution (last year's report before implementation). The patient was a dual-mode SimpliVity CN-3400.



Each node is a Dell R720 dual-bay rack server with a SimpliVity proprietary front panel and a special OmniStack accelerator card (OAC) PCIe card. Each server contains 2 E5-2680v3 chips, 24 16GB DDR3 strips and 24 disks (4 SSD 400GB, 20 HDD 1TB 7.2k SAS).



On the front, under the front panel, there are 24 disks - 4 SSDs and 20 2.5-inch SAS disks. On the reverse side are two power connectors, five network ports, the service port of the OAC card and two system HDD 300GB 10K SAS.


SimpliVity CN-3400, rear view


Omnistack Acccelerator Card on both sides

In a typical case, each server is connected by two 10 Gbps data connections, two 1 Gbps to the management network and one interface is required for the IPMI Server Management Console (iDRAC). This is just one of the connection options. You can not connect to a gigabit network, but use 10GbE both for a cluster network and for VM traffic. You can also add network cards and highlight several additional ports. In the case of a two-year configuration in the organization of a 10-gigabit network, there is no need at all - just connect the nodes directly to each other with two DAC cables.

After the power, network, and IP address configuration are configured on the IPMI console, you can proceed to the initial system configuration and integration of the solution into the vCenter.

In our case, vCenter was installed on a dedicated Windows Server. The vCenter Server Appliance is also supported.

We managed to test several ways to initialize the system. This is due to the fact that we first tested the kit in Russia (I remind you that the test was last year). And it was installed version of the system generation 2.x. In the case of this version, the procedure was as follows. The installed vCenter requires additional preparation: you need to add a new svtuser user and configure virtual switches. All settings are described in the brochure attached to the system.
After following the recommendations on how to prepare the existing infrastructure, you should set the IP addresses of the system, connect the nodes to the vCenter, create the SimpliVity federation, and initialize the VMware cluster. Everything is done using a simple wizard that comes with the system on a regular flash drive. During the setup, an arbitrator will be installed, designed to select the main node in the case of splitbrain. It can be installed both on the vCenter host and on a separate server.

After upgrading the system to the latest version (3.5.x), we tried to initialize the system again. In the latest versions, the installation is done using the Deployment Manager, he only needs access to the vCenter, he does the rest.
This completes the pre-configuration and in the vCenter - SimpliVity federation tab you can create the necessary datastores, backup schedules and use the rest of the functionality.

In general, setting up the system is quite simple and does not cause difficulties if the attached instructions are correctly followed. If there is no need to update the delivered system (IP addresses are prepared in advance and vCenter is configured), then the deployment of the system, including rack mounting, switching and initial configuration, will take about 30 minutes.
In the SimpliVity Federation section of vCenter, you can monitor performance, physical / logical capacity, deduplication and compression ratios, and manage backup policies.



The backup interface is similar to snapshots, but at the technical level, we have a single deduplicated space, and when creating a backup we have a fork of the virtual machine. There are backup integration modules with SQL and Exchange.
The deduplication factor, in our opinion, is not quite informative for the primary copy. For the rest is more or less adequate.

After creating a snapshot from it, you can restore both the entire VM and a separate file (with granularity, only Windows virtual machines are restored to files). When restoring a virtual machine, you can select “replace existing virtual machine” or “create a new one”. In this case, any backup can be manually restored in any remote data center.

All basic functions work correctly and without errors. SimpliVity seamlessly and tightly integrated into the vSphere client, and finding a particular function is easy.

Performance

VMware IO Analyzer 1.6.4 with virtual machine parameters was used to measure performance: VM version: 7, vCPU: 1. Memory: 2 GB, VD: 16 GB; OS: SUSE Linux Enterprise 11. On average, the result was 7,560 IOPS under load (8K, 50 / 50w / r, NoRandom), 20,352 IOPS for AllRandom. This is all past the cache.

The system also successfully survived all our favorite crash tests with the removal of disks on the go, interruption of interconnects and disconnection of nodes, and when returning components correctly restored their state. The impact on system performance with all our bullying was minimal, not counting the disconnection of interconnect. He helped speed up the system 2 times! (But having lost synchronous replication between nodes).

Conclusion: the piece of iron is interesting, as it makes it possible to replace almost all elements of the traditional data center architecture with a set of rack servers and switches. The system copes well with distributed loads, but is poorly suited for consolidated. No single-threaded applications or large databases.

Result in the store

The implementation took 2 days, one day to the site. Elaboration of the solution and testing took six months. Most of the time did the delivery of equipment and run tests. The customer considered various products that promise “infrastructure in a box”: hyperconvergent systems and a modular platform. We stopped at SimpliVity, because we do not need separate software for backup, all the machines can be seen from a single center - there is generally all control, and not a scattering of consoles, a very good relationship to the slop channel due to advanced deduplication. For example, with a daily full backup VM with 220 GB of used capacity, 570 MB flies through the communication channel. A full backup of the entire infrastructure is completed in 2-3 hours. For admins, it sounded like science fiction, but it actually works now.

The IT department of a small, dedicated experts on virtualization or backup, as in large companies, no. Therefore, employees have to deal with everything: from working with invoices to accompanying cash desks. There are no resources to study all the trends and nuances of storage, SAN, servers, backup software. As a result, the customer standardized the infrastructure and simplified his life. Instead of purchasing, setting up and maintaining servers, storage and backup in each store, he puts in a ready-made “cube” that he can run, store and backup virtual machines efficiently. One task in IT has become less. This is the meaning of hyperconvergence.

The result: two servers are put in the store, backup is set up in 5 minutes. Everything.

PS Thanks “Marvel Distribution” for providing test equipment. They supplied combat systems for the customer.