InfiniBox F2230 Review
Recently, ONLANTA acquired the InfiniBox F2230 system from Infinidat for its Oncloud.ru corporate cloud. There are almost no reviews of this system on the Internet, so we decided to eliminate the gap. Today we will try to tell you as much as possible about this storage system.
Infinidat is a young Israeli company. It was founded in 2011 by Moshe Yanay, who at one time led the development of EMC Symmetrix, and later founded the company XIV and was the chief architect of this system. By the way, XIV in 2008 was acquired by IBM. So the experience of Infinidat in the production of Enterprise-level storage systems does not hold. One of the key factors of Infinidat systems is their high reliability (99.99999%). Infinidat is the leader among disk array visionaries, according to Gartner analysts.
The F2230 series system takes up only 18U. At the same time, peak power consumption is only 3.5 kW, which is quite economical with a useful capacity of 250TB. This system fully supports compression, which can increase the effective volume to 1PB.
')
The main development of the company is an operating system that allows you to manage equipment with high quality. In essence, this is Software Defined Storage (SDS). These systems, which are now actively fighting for the market with their "classic" competitors. Infinidat is delivered as a complete system with all hardware components and its own rack.
Unload and unpack
The heart of the system is three servers operating in Active - Active - Active mode.
The use of three controllers allows you to reduce performance losses in case of failure of one of the controllers. In the classical two-year configuration, in the event of a single controller failure, the system is unable to maintain the write cache. This leads to its forced discharge, which in turn leads to an increase in delays. In the case of the three-preset configuration, if one of the controllers fails, the cache continues to synchronize between the remaining two nodes.
Controllers and BBU
The total amount of RAM of all controllers is 384GB. Also, our system has 24TB flash cache and two disk shelves, 60 NL-SAS disks each.
Install disks in the shelves
Disk shelves front view
Disk shelves, rear view
Each of the nodes in the system is equipped with several SSDs, which act as a global cache for all nodes (the data block is not cached twice). The address space of the volume is broken into pieces, the pieces are distributed among the controllers, each controller serves its own pieces and caches them. When the controller receives a request not for its piece, it takes it over the InfiniBand from another controller.
The connection between the nodes is done using InfiniBand. Disk shelves are connected to controllers via SAS. Data access can be organized both by FC, and using Ethernet.
InfiniRAID - the logical architecture of Infinidat uses small blocks of (64k) disks as RAID elements (instead of the entire disk). This minimizes disk rivalry and allows each of the nodes in the system to constantly use all disks for maximum performance.
As data enters the system, they are placed in RAM (for quick access), together with the DIF (Data Integrity Field) field, which remains with data at all levels of storage and processing, to prevent logical errors and lost records. The DIF is read along with the data during each read operation and is checked to detect all non-detectable errors (Silent-Data Corruption), which are then corrected using RAID.
When the data distribution algorithm decides to write data from memory to permanent media, it selects the blocks to write and optimizes them before that. This includes organizing blocks into groups of 14 sections (together with a DIF) + 2 parity blocks.
The algorithm distributes these groups (stripes) to all the disks in InfiniBox, while checking that:
This distribution of data, where each virtual RAID group includes partitions from different disk groups, allows you to use all system disks at the same time as data recovery. Therefore, InfiniBox can recover large disks in minutes instead of days. Due to the sufficient space allocated to protect against disk failure (12 disks in the older model), and using the ability to take unallocated space for protection, the system can withstand 12 broken disks and more.
Most reads come from cache located on SSDs. This allows you to minimize the load on NL-SAS disks, referring to them only for a small amount of data that did not fall into the cache.
Write operations go to DRAM-protected cache, which contains all the changes to the blocks before they “cool down”, after which they are already sent to the disks. When data blocks are sent to disks, they are written to all disks simultaneously, which gives the system the ability to record them at high speed. The data blocks that the system considers most probable for subsequent reading are recorded not only on NL-SAS disks, but also on the SSD cache to speed up reading.
This unique data layout allows InfiniBox to avoid using SAS drives and instead use a combination of SSD and NL-SAS drives to achieve high performance and low IO delays.
Installed, connected and configured system is ready for operation.
It is worth noting that this is a system that does a large number of tasks for users: it does not give access to disks — RAID has already been done in advance, load balancing on disks and between controllers is done automatically, controller firmware updates support, etc.
This icon in the upper right corner indicates that you have established a connection with the Infinidat server and support will be able to access the system for some kind of diagnostics, etc. When a support person connects to your array, the entire icon will turn green. Access for support can be disabled and enabled only when needed.
I would like to start with the fact that these systems do not support deduplication. At all. Only compression, which I will talk about in more detail.
To prevent data compression from affecting system performance, Infinibox compresses them at the time data is transferred from cache to disk. At present, the system uses the LZ4 compression algorithm, but the storage system architecture is designed in such a way that in the future it can be replaced with some newer and more efficient one. Do not forget that data compression is a processor-dependent operation, and even with modern processors it can affect the overall system performance. Compression is performed in blocks of 64KB, which allows for a higher compression ratio to be achieved.
Compression works with both thick and thin volumes, but its work is different in these cases.
So, how InfiniBox saves disk space without affecting performance: the data is stored in the RAM cache in uncompressed form. Compression occurs when data is written to disk. But it is recording on HDD, since what gets on the SSD cache is also there in uncompressed form. Infinidat assumes that most of the data you are working with is in the cache, so compression will not increase the response time. In fact, we understand that this is far from always the case, but if you look at the overall architecture, which involves storing data on a large number of disks in small blocks, this avoids a strong increase in response.
If you are interested in learning more about the operation of this system, I suggest referring to the article “ Storage Architecture for a Digital Enterprise ” by Vasily Kostryukov, Technical Director of Infinidat Russia.
InfiniBox supports both synchronous and asynchronous replication. Asynchronous replication is carried out on the basis of snapshots. Synchronous replication sends I / O to both systems (main and backup) and this increases response time, because The host receives confirmation of the data recording only after recording on both systems. Reading is only from the local system.
Asynchronous replication
If, in the opinion of the system, the quality of communication between the systems will not satisfy the requirements of synchronous replication, which is already configured, it will automatically transfer it to asynchronous mode and return it to synchronous mode when the proper connection quality is restored.
Asynchronous replication is supported on:
Synchronous replication to:
Operations that are not supported on replicated entities:
In synchronous replication mode is supported by changing the direction of replication.
To create a link for asynchronous replication, you will need four ports on each system (three data ports and one port for control), while for synchronous replication, you will need seven ports (six data ports and one control port).
Infinidat Host PowerTools for VMware allows you to enable virtual environment administrators to independently manage the allocation of disk resources. This makes it easier and saves time spent on basic operations. Host PowerTools also supports VASA (vStorage APIs for Storage Awareness).
Literally, on March 1, a plug-in for integrating this backup system with Infinibox arrays appeared on the Veeam website, which was developed by Infinidat in collaboration with Veeam, which gives hope for the high quality of this product. This plugin will allow using consistent snapshots of storage systems to increase backup speed and reduce the impact of backups on the production environment.
I will depart from the tradition of making a lot of screenshots of the interface and this time I made a video that I accompanied with the necessary comments.
InfiniBox systems will seamlessly integrate into any cloud infrastructure management system due to its API, which allows you to quickly and conveniently work with the system. It does not matter whether you provide services and resources are allocated using the control panel or the system serves only your own infrastructure and you need to quickly and conveniently add a large number of elements. All this is perfectly implemented using the API. There is also an InfiniSDK for Python .
Add 1000 hosts? No problem!
The other day, even the portal code.infinidat.com appeared, where you can find various scripts for integration with third-party systems (Openstack and Saltstack), modules for Ansible and many more interesting things. So if you are faced with the task of integrating this array, not just as part of the physical infrastructure, but convenient management, billing, etc., you will find a lot of useful information on this portal.
Considering that the ONLANTA cloud of ONLANTA company provides capacity for corporate customers under high loads (such as ERP, CRM, databases, etc.), has a developed architecture and develops its own capacity management tools, the InfiniBox F2230 system is perfectly suited for our infrastructure.
Infinidat is a young Israeli company. It was founded in 2011 by Moshe Yanay, who at one time led the development of EMC Symmetrix, and later founded the company XIV and was the chief architect of this system. By the way, XIV in 2008 was acquired by IBM. So the experience of Infinidat in the production of Enterprise-level storage systems does not hold. One of the key factors of Infinidat systems is their high reliability (99.99999%). Infinidat is the leader among disk array visionaries, according to Gartner analysts.
The F2230 series system takes up only 18U. At the same time, peak power consumption is only 3.5 kW, which is quite economical with a useful capacity of 250TB. This system fully supports compression, which can increase the effective volume to 1PB.
')
The main development of the company is an operating system that allows you to manage equipment with high quality. In essence, this is Software Defined Storage (SDS). These systems, which are now actively fighting for the market with their "classic" competitors. Infinidat is delivered as a complete system with all hardware components and its own rack.
The heart of the system is three servers operating in Active - Active - Active mode.
The use of three controllers allows you to reduce performance losses in case of failure of one of the controllers. In the classical two-year configuration, in the event of a single controller failure, the system is unable to maintain the write cache. This leads to its forced discharge, which in turn leads to an increase in delays. In the case of the three-preset configuration, if one of the controllers fails, the cache continues to synchronize between the remaining two nodes.
The total amount of RAM of all controllers is 384GB. Also, our system has 24TB flash cache and two disk shelves, 60 NL-SAS disks each.
Each of the nodes in the system is equipped with several SSDs, which act as a global cache for all nodes (the data block is not cached twice). The address space of the volume is broken into pieces, the pieces are distributed among the controllers, each controller serves its own pieces and caches them. When the controller receives a request not for its piece, it takes it over the InfiniBand from another controller.
The connection between the nodes is done using InfiniBand. Disk shelves are connected to controllers via SAS. Data access can be organized both by FC, and using Ethernet.
InfiniRAID - the logical architecture of Infinidat uses small blocks of (64k) disks as RAID elements (instead of the entire disk). This minimizes disk rivalry and allows each of the nodes in the system to constantly use all disks for maximum performance.
As data enters the system, they are placed in RAM (for quick access), together with the DIF (Data Integrity Field) field, which remains with data at all levels of storage and processing, to prevent logical errors and lost records. The DIF is read along with the data during each read operation and is checked to detect all non-detectable errors (Silent-Data Corruption), which are then corrected using RAID.
When the data distribution algorithm decides to write data from memory to permanent media, it selects the blocks to write and optimizes them before that. This includes organizing blocks into groups of 14 sections (together with a DIF) + 2 parity blocks.
The algorithm distributes these groups (stripes) to all the disks in InfiniBox, while checking that:
- each section on another disk
- sections are evenly distributed between the shelves (to protect against shelf problems),
- sections are evenly distributed between all disks (for optimal distribution of capacity and performance).
This distribution of data, where each virtual RAID group includes partitions from different disk groups, allows you to use all system disks at the same time as data recovery. Therefore, InfiniBox can recover large disks in minutes instead of days. Due to the sufficient space allocated to protect against disk failure (12 disks in the older model), and using the ability to take unallocated space for protection, the system can withstand 12 broken disks and more.
Most reads come from cache located on SSDs. This allows you to minimize the load on NL-SAS disks, referring to them only for a small amount of data that did not fall into the cache.
Write operations go to DRAM-protected cache, which contains all the changes to the blocks before they “cool down”, after which they are already sent to the disks. When data blocks are sent to disks, they are written to all disks simultaneously, which gives the system the ability to record them at high speed. The data blocks that the system considers most probable for subsequent reading are recorded not only on NL-SAS disks, but also on the SSD cache to speed up reading.
This unique data layout allows InfiniBox to avoid using SAS drives and instead use a combination of SSD and NL-SAS drives to achieve high performance and low IO delays.
It is worth noting that this is a system that does a large number of tasks for users: it does not give access to disks — RAID has already been done in advance, load balancing on disks and between controllers is done automatically, controller firmware updates support, etc.
This icon in the upper right corner indicates that you have established a connection with the Infinidat server and support will be able to access the system for some kind of diagnostics, etc. When a support person connects to your array, the entire icon will turn green. Access for support can be disabled and enabled only when needed.
Storage efficiency
I would like to start with the fact that these systems do not support deduplication. At all. Only compression, which I will talk about in more detail.
To prevent data compression from affecting system performance, Infinibox compresses them at the time data is transferred from cache to disk. At present, the system uses the LZ4 compression algorithm, but the storage system architecture is designed in such a way that in the future it can be replaced with some newer and more efficient one. Do not forget that data compression is a processor-dependent operation, and even with modern processors it can affect the overall system performance. Compression is performed in blocks of 64KB, which allows for a higher compression ratio to be achieved.
Compression works with both thick and thin volumes, but its work is different in these cases.
- Thin volumes — space saving from data compression remains available in the physical capacity of the pool.
- Thick volumes — space thus freed up is not available for re-allocation.
So, how InfiniBox saves disk space without affecting performance: the data is stored in the RAM cache in uncompressed form. Compression occurs when data is written to disk. But it is recording on HDD, since what gets on the SSD cache is also there in uncompressed form. Infinidat assumes that most of the data you are working with is in the cache, so compression will not increase the response time. In fact, we understand that this is far from always the case, but if you look at the overall architecture, which involves storing data on a large number of disks in small blocks, this avoids a strong increase in response.
If you are interested in learning more about the operation of this system, I suggest referring to the article “ Storage Architecture for a Digital Enterprise ” by Vasily Kostryukov, Technical Director of Infinidat Russia.
Replication
InfiniBox supports both synchronous and asynchronous replication. Asynchronous replication is carried out on the basis of snapshots. Synchronous replication sends I / O to both systems (main and backup) and this increases response time, because The host receives confirmation of the data recording only after recording on both systems. Reading is only from the local system.
- Host sends I / O to write to source InfiniBox
- Source InfiniBox confirms host I / O record
- Source InfiniBox replicates data to target InfiniBox
- Target InfiniBox confirms replication to source InfiniBox
If, in the opinion of the system, the quality of communication between the systems will not satisfy the requirements of synchronous replication, which is already configured, it will automatically transfer it to asynchronous mode and return it to synchronous mode when the proper connection quality is restored.
Asynchronous replication is supported on:
- tomash
- file systems
- consistency groups.
Synchronous replication to:
- Tomah.
Operations that are not supported on replicated entities:
- removal,
- recovery,
- change of size,
- disabling write-protect,
- file system export.
In synchronous replication mode is supported by changing the direction of replication.
To create a link for asynchronous replication, you will need four ports on each system (three data ports and one port for control), while for synchronous replication, you will need seven ports (six data ports and one control port).
Application Integration
Infinidat Host PowerTools for VMware allows you to enable virtual environment administrators to independently manage the allocation of disk resources. This makes it easier and saves time spent on basic operations. Host PowerTools also supports VASA (vStorage APIs for Storage Awareness).
Literally, on March 1, a plug-in for integrating this backup system with Infinibox arrays appeared on the Veeam website, which was developed by Infinidat in collaboration with Veeam, which gives hope for the high quality of this product. This plugin will allow using consistent snapshots of storage systems to increase backup speed and reduce the impact of backups on the production environment.
Interface, RESTful API and SDK
I will depart from the tradition of making a lot of screenshots of the interface and this time I made a video that I accompanied with the necessary comments.
InfiniBox systems will seamlessly integrate into any cloud infrastructure management system due to its API, which allows you to quickly and conveniently work with the system. It does not matter whether you provide services and resources are allocated using the control panel or the system serves only your own infrastructure and you need to quickly and conveniently add a large number of elements. All this is perfectly implemented using the API. There is also an InfiniSDK for Python .
Add 1000 hosts? No problem!
#!/usr/bin/python #! -*- coding: utf-8 -*- from infinisdk import InfiniBox from infi.dtypes.iqn import make_iscsi_name datastring = ''' iqn.1990-06.vmware:host1 … iqn.1991-06.vmware:host1000 ''' system = InfiniBox('infinidat_ip', auth=('infinidat_login', 'infinidat_password')) system.login() datastring = datastring.splitlines() for i, iqn in enumerate(datastring): host_name = 'host-' + str(i) host = system.hosts.create(name=host_name) address = make_iscsi_name(iqn) host.add_port(address) The other day, even the portal code.infinidat.com appeared, where you can find various scripts for integration with third-party systems (Openstack and Saltstack), modules for Ansible and many more interesting things. So if you are faced with the task of integrating this array, not just as part of the physical infrastructure, but convenient management, billing, etc., you will find a lot of useful information on this portal.
Considering that the ONLANTA cloud of ONLANTA company provides capacity for corporate customers under high loads (such as ERP, CRM, databases, etc.), has a developed architecture and develops its own capacity management tools, the InfiniBox F2230 system is perfectly suited for our infrastructure.
By the way, our company has a number of interesting vacancies.
Source: https://habr.com/ru/post/350270/
All Articles