NetApp SDS: ONTAP Select
ONTAP Select is a logical evolution of the Data ONTAP-v line , i.e. Software Defined Storage. ONTAP software (an operating system or firmware in a traditional way, if you wish) can be used on a specialized FAS hardware platform or as a virtual machine: in public clouds or on a chest of drawers equipment.
The last two options are called ONTAP for Cloud and ONTAP Select, respectively.
Like the predecessor of ONTAP Select, this product, which lives in the form of a virtual machine and fully relies on the traditional RAID controller installed in your server. NAS (CIFS, NFS) and IP SAN (iSCSI) protocols are supported and FCP is not supported. In the documents NetApp can be found the internal name ONTAP Select - sDOT, this is the same thing.
From the expected innovations:
Along with High Availability and Clustering, single-anode configurations are still supported.
To provide the High Availability functionality, SyncMirror technology is used (the same as in MetroCluster) which works like RAID1: there are two plexes, one main and one backup. From the main there is a reading and record, and in a reserve there is a synchronous replica. If one of the two nodes in the HA configuration with the main plex dies, the second node uses the backup plex, the applications switch transparently and do not notice the loss of the node. In the figure, each node has two data sets (two plexes): its own (main) and neighbor (backup).
')
ONTAP Select can act as a source, and as the recipient of replicas. Based on SnapMirror, you can build a DR solution SVM . If ONTAP is used only as a file repository, then there is no need for replication to provide data consistency with third-party software. If you use IP SAN or virtualization applications, databases and others, then backup software will be needed to ensure the consistency of the information.
The easiest way to take a consistent snapshot is to put out the application. What is data consistency? Consistency refers to backup requirements that guarantee data recovery at the time of backup. This is if explained in a dry, completely technical language. And if it is simple and understandable, then it is better explained and understood from the opposite: Imagine that you have a database that lives in a virtualization environment or on storage systems that can do snapshots and they are set up, so they are removed every day at 12 at night. Can you recover to one of such snapshots if it is damaged? The answer is “maybe yes or maybe not”, even “no more than yes”. Because by themselves snapshots without any interaction with the application to sculpt, regardless of whether the database transactions have completed and all the data from the memory were on the disk. In other words, you need to somehow integrate and “make friends” of your snaps and your application, so that it flushes data from the cache to disk, completes all accumulated transactions, stops, followed by snapshot, and only at the end the application “died”. And so that this happens unnoticed by users, snapshot must be removed instantly. And only under this condition, you are guaranteed to be able to roll back to such an inappropriate one.
All modern hypervisors, with the help of agents installed inside virtual machines, can achieve consistency and continue to perform consistent snapshots, this is very convenient for small companies that have not very loaded virtual machines. If you do not install the agent hypervisor, then the consistency will not.
For the time being, the organization and the load on the virtual machines are not large, consistent snepshots of hypervisors and backup solutions built on this will be suitable for such companies. But companies and workloads can grow, and here comes a turning point:
Hyper-V snapshots or the same VMware are arranged, unfortunately in such a way that the more they exist, the more this snapshot becomes. The more snapshots, the longer it will be removed. The more it is removed, the more it loads the disk subsystem. The longer it is removed, the greater the likelihood that the old snap will not have time to retire due to the increased additional load, and another new snap will be created for the new backup process. The more snapshots, the greater the load on the disk subsystem.
Both VMware and Microsoft openly say that the hypervisor snapshots are not recommended for use on highly loaded applications. I personally observed two times in practice that the process of removing a snapshot (consolidation) led to the collapse of the system, in both cases this was due to the fact that the disk subsystem did not have time to cope with the additional load, the block device increased and increased due to excessive load the response time has not yet exceeded the timeout and the virtual machine, having not yet managed to consolidate, has gone offline with its subsequent damage. Once even managed to restore it.
In general, at a certain point, it becomes clear that snapshots of hypervisors are not suitable as a backup paradigm.
SNPS made on ONTAP do not affect the performance of the storage system at all. This is the internal ONTAP architecture. But the storage snapshots are the storage snapshots. And so that they can interact with the hypervisor, you need to use some kind of specialized software solution for interaction. Examples of such solutions are Veeam B & R, vVOL technology and others. It is important to separately note the decision of the company NetApp SnapCreator, which is able to provide this bundle, since ONTAP Select is designed for not large organizations, but SnapCreator is a free product.
This is a free web-based framework for managing and integrating with ONTAP and various applications, including Hyper-V, VMware and KVM hypervisors. Due to its modularity, the framework can be integrated with many other software products with the help of plug-ins, and if necessary expanded with the help of scripts. Learn more about SnapCreator here and here .
Hypervisors
Equipment:
Licensing
Lossy byte. The maximum size is 400TB.
ONTAP Select includes licenses: CIFS, NFS, iSCSI, FlexClone, SnapRestore, SnapVault, SnapMirror, Virtual Storage Console.
The SnapCenter license is not included and is purchased for each individual host.
Distinctive features
The following is included in the basic delivery when buying ONTAP Select, in addition, this functionality does not need to be bought:
SDS ONTAP is well suited for medium and small companies, and branch companies, where it may be necessary to replicate data to other ONTAPs. The product allows clustering for online migration of your data across cluster nodes, has rich functionality and wide integration with a variety of applications and allows you to build high-level storage using a server comoditi. Backup products that integrate ONTAP with KVM and ESXi virtual environments will be in great demand in such schemes as they allow to improve the backup process qualitatively. So Veeam Backup & Replication and free SnapCreator allow you to perform consistent backups in conjunction with hypervisors and know how to manage SnapMirror replication, which makes these and other similar products, the best tandem in such an infrastructure. The SnapMirror for SVM technology will help build a DR solution based on ONTAP. Separately, it is worth highlighting VMware vVOL technology, which natively provides consistent snapshots on ONTAP because of which it will also be very interesting for such installations.
This may contain links to Habra articles that will be published later .
I ask to send messages on errors in the text to the LAN .
Comments, additions and questions on the article on the contrary, please in the comments .
The last two options are called ONTAP for Cloud and ONTAP Select, respectively.
Like the predecessor of ONTAP Select, this product, which lives in the form of a virtual machine and fully relies on the traditional RAID controller installed in your server. NAS (CIFS, NFS) and IP SAN (iSCSI) protocols are supported and FCP is not supported. In the documents NetApp can be found the internal name ONTAP Select - sDOT, this is the same thing.
From the expected innovations:
- High Avalability Support
- Clustering support up to 4 nodes
- Maximum useful capacity is 400 TB (100TB per node in 4 node cluster)
Along with High Availability and Clustering, single-anode configurations are still supported.
High availability
To provide the High Availability functionality, SyncMirror technology is used (the same as in MetroCluster) which works like RAID1: there are two plexes, one main and one backup. From the main there is a reading and record, and in a reserve there is a synchronous replica. If one of the two nodes in the HA configuration with the main plex dies, the second node uses the backup plex, the applications switch transparently and do not notice the loss of the node. In the figure, each node has two data sets (two plexes): its own (main) and neighbor (backup).
')
SnapMirror replication
ONTAP Select can act as a source, and as the recipient of replicas. Based on SnapMirror, you can build a DR solution SVM . If ONTAP is used only as a file repository, then there is no need for replication to provide data consistency with third-party software. If you use IP SAN or virtualization applications, databases and others, then backup software will be needed to ensure the consistency of the information.
Point-in-time consistency
The easiest way to take a consistent snapshot is to put out the application. What is data consistency? Consistency refers to backup requirements that guarantee data recovery at the time of backup. This is if explained in a dry, completely technical language. And if it is simple and understandable, then it is better explained and understood from the opposite: Imagine that you have a database that lives in a virtualization environment or on storage systems that can do snapshots and they are set up, so they are removed every day at 12 at night. Can you recover to one of such snapshots if it is damaged? The answer is “maybe yes or maybe not”, even “no more than yes”. Because by themselves snapshots without any interaction with the application to sculpt, regardless of whether the database transactions have completed and all the data from the memory were on the disk. In other words, you need to somehow integrate and “make friends” of your snaps and your application, so that it flushes data from the cache to disk, completes all accumulated transactions, stops, followed by snapshot, and only at the end the application “died”. And so that this happens unnoticed by users, snapshot must be removed instantly. And only under this condition, you are guaranteed to be able to roll back to such an inappropriate one.
Virtualization and consistency
All modern hypervisors, with the help of agents installed inside virtual machines, can achieve consistency and continue to perform consistent snapshots, this is very convenient for small companies that have not very loaded virtual machines. If you do not install the agent hypervisor, then the consistency will not.
Why are the hypervisor snapshots evil?
For the time being, the organization and the load on the virtual machines are not large, consistent snepshots of hypervisors and backup solutions built on this will be suitable for such companies. But companies and workloads can grow, and here comes a turning point:
Hyper-V snapshots or the same VMware are arranged, unfortunately in such a way that the more they exist, the more this snapshot becomes. The more snapshots, the longer it will be removed. The more it is removed, the more it loads the disk subsystem. The longer it is removed, the greater the likelihood that the old snap will not have time to retire due to the increased additional load, and another new snap will be created for the new backup process. The more snapshots, the greater the load on the disk subsystem.
Both VMware and Microsoft openly say that the hypervisor snapshots are not recommended for use on highly loaded applications. I personally observed two times in practice that the process of removing a snapshot (consolidation) led to the collapse of the system, in both cases this was due to the fact that the disk subsystem did not have time to cope with the additional load, the block device increased and increased due to excessive load the response time has not yet exceeded the timeout and the virtual machine, having not yet managed to consolidate, has gone offline with its subsequent damage. Once even managed to restore it.
In general, at a certain point, it becomes clear that snapshots of hypervisors are not suitable as a backup paradigm.
Napshots ONTAP
SNPS made on ONTAP do not affect the performance of the storage system at all. This is the internal ONTAP architecture. But the storage snapshots are the storage snapshots. And so that they can interact with the hypervisor, you need to use some kind of specialized software solution for interaction. Examples of such solutions are Veeam B & R, vVOL technology and others. It is important to separately note the decision of the company NetApp SnapCreator, which is able to provide this bundle, since ONTAP Select is designed for not large organizations, but SnapCreator is a free product.
SnapCreator
This is a free web-based framework for managing and integrating with ONTAP and various applications, including Hyper-V, VMware and KVM hypervisors. Due to its modularity, the framework can be integrated with many other software products with the help of plug-ins, and if necessary expanded with the help of scripts. Learn more about SnapCreator here and here .
Supported ONTAP Select Configurations
Hypervisors
- VMware vSphere 5.X or 6.X, Enterprise or Enterprise Plus
- KVM hypervisor
Equipment:
- Intel Haswell CPU (Xeon E5 v3)
- ONTAP Select virtual machine: 4 virtual
- CPUs / 16GB memory
- Storage:
- SAS, NL-SAS disk support
- Hardware RAID controller with writeback cache
- Network: minimum 2 x 10GbE ports
Licensing
Lossy byte. The maximum size is 400TB.
ONTAP Select includes licenses: CIFS, NFS, iSCSI, FlexClone, SnapRestore, SnapVault, SnapMirror, Virtual Storage Console.
The SnapCenter license is not included and is purchased for each individual host.
Distinctive features
The following is included in the basic delivery when buying ONTAP Select, in addition, this functionality does not need to be bought:
- SAN (iSCSI) & NAS Support (CIFS & NFS)
- CIFS (SMB) is supported both in integration with AD and using Workgroup
- High availability
- Clustering with the possibility of online migration of SAN / NAS data across the cluster nodes
- Snapshots and clones that do not affect storage performance - FlexClone and SnapRestore licenses come as standard
- QoS - at the file level, moon, volyum
- MS ODX & Copy Offload (SAN / NAS), SMB 3.1
- NFS 4.X, pNFS
- NDMP with 0x2050 extension support
- Thing provisioning, SCSI-3 UNMAP (Space Reclamation)
- resource exhaustion reporting - host notification of the end of space in a thin moon
- Compression
- Deduplication
- Storage Virtual Machine (Multi Tenancy) - Ability to create dedicated “virtual” cluster-based storage systems that can work independently of each other and be managed by various administrators, departments or even companies
- Integration with advanced backup software
- Support vVol (NFS / iSCSI), VASA, VAAI (NFS / iSCSI)
- Ability to create Disastar Recovery solutions on multiple sites
- Support for fine data replication to remote ONTAP (no matter FAS, ONTAP Select or ONTAP for Cloud) - SnapMirror and SnapVault licenses come as standard
- Free SnapCreator - software integration snaps with applications (database, virtualization, etc.)
- Free OnCommand Unified Manager & Performance Manager - software for monitoring performance, security and fullness of storage
findings
SDS ONTAP is well suited for medium and small companies, and branch companies, where it may be necessary to replicate data to other ONTAPs. The product allows clustering for online migration of your data across cluster nodes, has rich functionality and wide integration with a variety of applications and allows you to build high-level storage using a server comoditi. Backup products that integrate ONTAP with KVM and ESXi virtual environments will be in great demand in such schemes as they allow to improve the backup process qualitatively. So Veeam Backup & Replication and free SnapCreator allow you to perform consistent backups in conjunction with hypervisors and know how to manage SnapMirror replication, which makes these and other similar products, the best tandem in such an infrastructure. The SnapMirror for SVM technology will help build a DR solution based on ONTAP. Separately, it is worth highlighting VMware vVOL technology, which natively provides consistent snapshots on ONTAP because of which it will also be very interesting for such installations.
This may contain links to Habra articles that will be published later .
I ask to send messages on errors in the text to the LAN .
Comments, additions and questions on the article on the contrary, please in the comments .
Source: https://habr.com/ru/post/303146/
All Articles