About SAN (Storage Area Network) on the fingers

In the matter of knowledge, SAN has encountered a certain obstacle - the inaccessibility of basic information. In the matter of studying other infrastructure products that have been encountered, it is easier - there are trial versions of software, the ability to install them on a virtual machine, there are a lot of textbooks, reference guides and blogs on the topic. Cisco and Microsoft rivet very high-quality textbooks, MS in addition at the very least combed his hellish attic storage room called technet, even in VMware there is a book, albeit one (and even in Russian!), And with an efficiency of about 100%. Already on the data storage devices you can get information from seminars, marketing events and documents, forums. On the same network of storage - silence and dead with braids to stand. I found two textbooks, but did not dare to buy. This " Storage Area Networks For Dummies " (there is such a thing, it turns out. Very curious English-speaking "teapots" in the target audience, apparently) for one and a half thousand rubles and " Distributed Storage Networks: Architecture, Protocols and Management " - looks more reliable, but $ 8,200 at a discount of 40%. Together with this book, Ozon also recommends the book The Art of Brickwork.

What advice to a person who decides to study from scratch at least the theory of storage networking, I do not know. As practice has shown, even expensive courses can give zero on the way out. With regard to SAN, people fall into three categories: those who do not know what it is, who know what such a phenomenon is, and those who look at the question “why do two or more factories in a storage network” look with such confusion as if they were asked something like "why square four corners?".
')
I will try to fill the gap that I lacked - to describe the base and describe it simply. I will consider SAN based on its classic protocol - Fiber Channel.

So, SAN - Storage Area Network - is designed to consolidate server disk space on dedicated disk storage. The bottom line is that disk resources are more economical, easier to manage, and have greater performance. And in matters of virtualization and clustering, when several servers need access to a single disk space, such storage systems are generally indispensable.

By the way, in the terminology of SAN, due to the translation into Russian, there is some confusion. SAN in translation means "data storage network" - DSS. However, classically in Russia, storage means the term “storage system”, that is, a disk array ( Storage Array ), which in turn consists of a Control Unit ( Storage Processor, Storage Controller ) and disk shelves ( Disk Enclosure ). However, in the original Storage Array is only part of the SAN, although sometimes the most significant. In Russia, we find that the storage system (data storage system) is part of the storage system (storage network). Therefore, storage devices are usually referred to as storage, and storage network - SAN (and confused with the "Sun", but this is the little things).

Components and terms

Technologically, a SAN consists of the following components:

1. Nodes, nodes

• Disk arrays (data storage systems) - storages (targets [targets])
• Servers are consumers of disk resources (initiators).

2. Network infrastructure

• Switches (and routers in complex and distributed systems)
• Cables

Special features

Without going into details, the FC protocol is similar to the Ethernet protocol with WWN addresses instead of MAC addresses. Only, instead of two levels, Ethernet has five (of which the fourth is not yet defined, and the fifth is the mapping between the FC transport and high-level protocols that are transmitted over this FC - SCSI-3, IP). In addition, FC switches use specialized services, analogs of which for IP networks are usually hosted on servers. For example: Domain Address Manager (responsible for assigning Domain ID to switches), Name Server (stores information about connected devices, a kind of WINS analog within the switch), etc.

For SAN, the key parameters are not only performance, but also reliability. After all, if the database server lost the network for a couple of seconds (or even minutes) - well, it will be unpleasant, but you can survive. And if at the same time the hard drive with the base or the OS falls off, the effect will be much more serious. Therefore, all SAN components are usually duplicated - ports in storage devices and servers, switches, links between switches and, a key feature of SAN, compared to LAN - duplication at the level of the entire infrastructure of network devices - fabric.

A factory ( fabric - which is actually translated from English as a cloth, since the term symbolizes an interlaced connection scheme of network and end devices, but the term has already been settled) - a set of switches interconnected by inter-switch links ( ISL - InterSwitch Link ).

Highly reliable SANs necessarily include two (and sometimes more) factories, because the factory itself is a single point of failure. Those who have at least once observed the effects of a ring on the network or deft keyboard movement that introduces a kernel-level switch or distribution to an unsuccessful firmware or command understand what they mean.

Factories may have an identical (mirror) topology or vary. For example, one factory can consist of four switches, and the other from one, and only highly critical nodes can be connected to it.

Topology

The following types of factory topologies are distinguished:

Cascade switches are connected in series. If there are more than two, then it is unreliable and unproductive.

Ring - closed cascade. More reliably just a cascade, although with a large number of participants (more than 4), performance will suffer. A single failure ISL or one of the switches turns the circuit into a cascade with all the consequences.

Mesh There is Full Mesh - when each switch connects to each. Characterized by high reliability, performance and price. The number of ports required for inter-switch communications, with the addition of each new switch to the circuit, grows exponentially. With a certain configuration, there will simply be no ports left for the nodes - everyone will be busy under ISL. Partial Mesh - any chaotic combination of switches.

The center / periphery (Core / Edge) is close to the classical LAN topology, but without a distribution level. Often, storages are connected to Core switches, and servers are connected to Edge. Although an additional layer (tier) of edge switches can be allocated for storage. Also, storage and servers can be connected to one switch to improve performance and reduce response time (this is called localization). This topology is characterized by good scalability and manageability.

Zoning (zoning)

Another characteristic of SAN technology. This is the definition of initiator-target pairs. That is, which servers can access to which disk resources so that it does not happen that all servers see all possible disks. This is achieved as follows:

• selected pairs are added to zones previously created on the switch;
• zones are placed in zone sets (zone set, zone config) created there;
• zone sets are activated in the factory.

For the initial post on SAN, I think that's enough. I apologize for the variegated pictures - there is no possibility to draw oneself at work while there is no time at home. There was a thought to draw on paper and take a picture, but decided it was better that way.

Finally, as a postscript, I will list the basic guidelines for designing a SAN factory .

• Design the structure so that between two end devices there are no more than three switches.
• It is desirable that the factory consisted of no more than 31 switches.
• It is necessary to set the Domain ID manually before introducing a new switch into the factory - it improves manageability and helps to avoid problems with the same Domain ID, in cases, for example, reconnecting the switch from one factory to another.
• Have several equal routes between each storage device and initiator.
• In cases of uncertain performance requirements, proceed from the ratio of the number of Nx-ports (for end devices) to the number of ISL-ports as 6: 1 (EMC recommendation) or 7: 1 (Brocade recommendation). This ratio is called oversubscription.
• Recommendations for zoning:
  - use informative names of zones and zones-sets;
  - use WWPN zoning, not Port-based (based on device addresses, not physical ports of a specific switch);
  - each zone - one initiator;
  - to clean the factory from the "dead" zones.
• Have a reserve of free ports and cables.
• Have a reserve of equipment (switches). At the site level - necessarily, perhaps at the factory level.