Anarchy and Data Storage: Does SAN Have a Future?
Translator's note: Not so long ago, Robin Harris, an expert at ZDnet, published an article about “ why SSDs have become outdated ” (an adapted version of it will be released in our blog soon). Another expert of the publication, Jason Perlow, decided, pushing off from this article, to speculate about the disadvantages and prospects of network storage systems (SAN and NAS) - he believes that the future of the approach of cloud providers and the use of JBOD.
The material presented below contains a significant number of technical terms, the translation of which can lead to inaccuracies. If you notice a typo, mistake or inaccuracy of the translation - write to us and we will fix everything promptly.
')
It is important to understand that the currently used SAS and SATA interfaces are nothing more than evolved SCSI and ATA technologies that have evolved over the decades. Since then, server technologies have gone far ahead, and the need for data processing applications has increased significantly.
It is obvious that representatives of the industry should think about the further modernization of existing data storage technologies. However, the task of squeezing out more IOPS from an SSD is more relevant to science, from a practical point of view where the total cost of ownership of storage media is more important.
In large companies, the issue of organizing data storage is becoming ever more acute - especially in regulated industries, whose members are obliged to store data for a long time according to legal regulations. In addition, this information should be easily accessible - this complicates the use of archiving technologies.
To put it bluntly, SAN and NAS cost a lot of money, and they take up to a third of all the hardware costs in data centers of companies. Yes, manufacturers of such products have earned an excellent reputation for themselves, so their products are used to store the most critical data, but let's be honest - there is no magic in these boxes filled with discs the size of a refrigerator.
Inside all the same SATA, SAS, various interfaces, controllers and special software, which is responsible for creating logical device numbers (LUN). Most of these controllers run on proprietary versions of UNIX or even BSD variants — the user will never see any of this. For him, a SAN or NAS after creating a LUN is a real black box.
This expensive insanity can be stopped, but this requires a non-standard and creative approach from representatives of large businesses - this means that they must treat the storage as well as the providers of scalable services (hyperscale). When we talk about scaling, we mean Amazon Web Services, Microsoft Azure, or Google Computer Engine.
Do you think that these companies managed to create cloud storage systems exclusively using EMC and NetApp hardware?
Of course not, it’s just impossible. Instead of SAN and NAS, such companies use JBOD - arrays of “ just disks ” (Just a Bunch of Disks). They were able to create a cheaper infrastructure with the help of affordable and affordable equipment, JBOD and experienced engineers.
There are no standards for building such infrastructures; moreover, this process is complicated by the fact that historically there is an opinion that it is necessary to use fiber to create clustered storages - and this is still very expensive.
However, thanks to Ethernet 10Gigabit and 40Gigabit networks, Ethernet RDMA cards and the advent of the SMB 3.0 network access protocol, the situation is changing rapidly.
The concept is quite simple - the organization simply connects many “heads” of file servers to the existing switched Ethernet infrastructure, and uses many JBODs (they are made, for example, DataOn, Dell or Supermicro) made up of SAS 15K and SSD in longline configuration and connected with these “heads” in the SAS cluster.
In turn, these anchor file servers are connected to virtualized or physical systems that provide access to data using SMB 3.0. The elasticity of such a system depends on the OS managing the storage, and not on some secret software embedded in the controllers, as is done in SAN and NAS.
In the scenario described in the image above, the Microsoft Scale-out File Server (SoFS) file servers are used that come with the built-in Windows Server 2012 R2 and use the Storage Spaces component to work. As an iron, DataOn DNS-1660D is used here in combination with Dell R820 rack servers and Mellanox RDMA cards.
The configuration described above can achieve a constant speed of more than 1 million IOPS per second.
A publication about building SoFS arrays of JBOD using MD1220 PowerVault released Dell. In general, any combination of JBOD, the common hardware of x86 architecture using SAS and 10 Gbps Ethernet connection will work.
In addition to Microsoft, there are other vendors involved in building architectures based on JBOD - for example, Nexenta (based on ZFS from Solaris), for Linux there are HA-LVM and GFS / GFS2, including the Red Hat Resilient Storage component . The equivalent for Ubuntu Server is called ClusterStack .
The conclusion here is that despite the reliability and reliability of the SAN and NAS solutions, their hegemony in ensuring the highest performance and “elasticity” of the storage facilities is coming to an end, and other tools will soon be widely distributed.
Managers of companies that are trying to save on storage an ever-increasing amount of data in the near future may resort to the method used by cloud providers - using JBOD and software defined storage ( about this, by the way, there was an article on Habré ) and the use of cloud-integrated storage ( CIS ) for applications that allow backups in the cloud.
The material presented below contains a significant number of technical terms, the translation of which can lead to inaccuracies. If you notice a typo, mistake or inaccuracy of the translation - write to us and we will fix everything promptly.
')
It is important to understand that the currently used SAS and SATA interfaces are nothing more than evolved SCSI and ATA technologies that have evolved over the decades. Since then, server technologies have gone far ahead, and the need for data processing applications has increased significantly.
It is obvious that representatives of the industry should think about the further modernization of existing data storage technologies. However, the task of squeezing out more IOPS from an SSD is more relevant to science, from a practical point of view where the total cost of ownership of storage media is more important.
In large companies, the issue of organizing data storage is becoming ever more acute - especially in regulated industries, whose members are obliged to store data for a long time according to legal regulations. In addition, this information should be easily accessible - this complicates the use of archiving technologies.
To put it bluntly, SAN and NAS cost a lot of money, and they take up to a third of all the hardware costs in data centers of companies. Yes, manufacturers of such products have earned an excellent reputation for themselves, so their products are used to store the most critical data, but let's be honest - there is no magic in these boxes filled with discs the size of a refrigerator.
Inside all the same SATA, SAS, various interfaces, controllers and special software, which is responsible for creating logical device numbers (LUN). Most of these controllers run on proprietary versions of UNIX or even BSD variants — the user will never see any of this. For him, a SAN or NAS after creating a LUN is a real black box.
This expensive insanity can be stopped, but this requires a non-standard and creative approach from representatives of large businesses - this means that they must treat the storage as well as the providers of scalable services (hyperscale). When we talk about scaling, we mean Amazon Web Services, Microsoft Azure, or Google Computer Engine.
Do you think that these companies managed to create cloud storage systems exclusively using EMC and NetApp hardware?
Of course not, it’s just impossible. Instead of SAN and NAS, such companies use JBOD - arrays of “ just disks ” (Just a Bunch of Disks). They were able to create a cheaper infrastructure with the help of affordable and affordable equipment, JBOD and experienced engineers.
There are no standards for building such infrastructures; moreover, this process is complicated by the fact that historically there is an opinion that it is necessary to use fiber to create clustered storages - and this is still very expensive.
However, thanks to Ethernet 10Gigabit and 40Gigabit networks, Ethernet RDMA cards and the advent of the SMB 3.0 network access protocol, the situation is changing rapidly.
The concept is quite simple - the organization simply connects many “heads” of file servers to the existing switched Ethernet infrastructure, and uses many JBODs (they are made, for example, DataOn, Dell or Supermicro) made up of SAS 15K and SSD in longline configuration and connected with these “heads” in the SAS cluster.
In turn, these anchor file servers are connected to virtualized or physical systems that provide access to data using SMB 3.0. The elasticity of such a system depends on the OS managing the storage, and not on some secret software embedded in the controllers, as is done in SAN and NAS.
In the scenario described in the image above, the Microsoft Scale-out File Server (SoFS) file servers are used that come with the built-in Windows Server 2012 R2 and use the Storage Spaces component to work. As an iron, DataOn DNS-1660D is used here in combination with Dell R820 rack servers and Mellanox RDMA cards.
The configuration described above can achieve a constant speed of more than 1 million IOPS per second.
A publication about building SoFS arrays of JBOD using MD1220 PowerVault released Dell. In general, any combination of JBOD, the common hardware of x86 architecture using SAS and 10 Gbps Ethernet connection will work.
In addition to Microsoft, there are other vendors involved in building architectures based on JBOD - for example, Nexenta (based on ZFS from Solaris), for Linux there are HA-LVM and GFS / GFS2, including the Red Hat Resilient Storage component . The equivalent for Ubuntu Server is called ClusterStack .
The conclusion here is that despite the reliability and reliability of the SAN and NAS solutions, their hegemony in ensuring the highest performance and “elasticity” of the storage facilities is coming to an end, and other tools will soon be widely distributed.
Managers of companies that are trying to save on storage an ever-increasing amount of data in the near future may resort to the method used by cloud providers - using JBOD and software defined storage ( about this, by the way, there was an article on Habré ) and the use of cloud-integrated storage ( CIS ) for applications that allow backups in the cloud.
Source: https://habr.com/ru/post/256845/
All Articles