Unified Storage - “the servant of two masters”: NAS + SAN

Unified Storage is called a “multi-protocol” storage system, such that when it is possible to work with data stored on it using different protocols within the same system. Unified storage is usually understood as a system that can work simultaneously in a SAN network, that is, with “block access” using FC or iSCSI protocols, and as a NAS device for storing files.

It all began in the early 2000s, when NetApp launched its new storage series called FAS - Fabric Attached Storage - something like “network-attached storage”, the fact is that the Fiber Channel infrastructure from switches, traditionally called Fabric, "factory".
Such systems could work as well-proven company NAS systems since 1994, that is, as network file storages using the NFS and CIFS protocols, but also as SAN storage devices, using the FibreChannel protocol (and soon they added IP -SAN "," block "iSCSI protocol).

This turnaround from NAS, a consistent supporter and manufacturer of NetApp by then, to a SAN that NetApp was not interested in at that time, and actively competed, was a surprise, as was the chosen “matching” model of these two “paradigms” in one device. No one did this, and NetApp was again the “inventor”, “innovator” and promoter of the new idea, initially, as usual, perceived “in hostility”.
However, time put everything in its place, and today Unified Storage calls its systems such “industry giants” as EMC and HP , which have been actively fighting against NetApp for a long time and against this idea.

However, as one of the great physicists said, the idea in the course of its life goes from “What a nonsense!” To “Well, that's obvious!”. A similar way has been done and the idea to combine in one storage device all the necessary protocols for accessing its data, both NAS and SAN.
')
Here's what Dave Hitz, co-founder of the company, wrote about how NetApp created its SANs in his book How to Neuter a Bull: Unexpected Lessons in Risk, Growth and Success in Business , on the company's history and experience in creating a second-generation startup. in the world manufacturer of storage systems. "

Suppose you always hear in the church "Jesus is our Lord and true God," and one day they say to you, "But Allah is also cool, we will give him his due." Here's the story, how I fell into some kind of heresy, and decided that NetApp should do and sell another SAN in addition to the NAS. Moreover, my companion and co-founder of NetApp, James Lau, was one of our first heretics! It may seem strange that I cite such “strong” analogies for some technical things, but it seems to me that the analogy is very close, and this process has always caused a lot of heated debate, even within our company, although we are in ourselves, as far as I it is known that no one was stoned, they were not crucified and did not blow up.
That's how it all happened.

NAS means Network-Attached Storage, “Network Storage,” while SAN stands for Storage Area Network, “Storage Network.” Based on the names, you can conclude that this is about the same thing - both of them are for storage, and both are of networks. There is a classic high-end / low-end situation. Initially, SAN was better suited for high-end users, but it was also significantly more expensive. The NAS was cheaper, grew rapidly and improved in its capabilities, and besides it also had great prospects for its growth and improvement.
Looking back, I see that there were some obvious motives for us to do SAN systems too. Yes, and some market analysts have advised us the same. Of course, the same analysts asserted that the growth of the NAS market would soon stop, even though we doubled our growth in this market every year, so we were skeptical about these prophecies.

Another sign that it was time for us to expand the capabilities of our systems was the story of one of our clients. Jason was our longtime supporter and supporter of the NAS concept. He liked NAS as a storage architecture, and he used them for storage always and everywhere, even if the application vendor recommended using SAN. But at one of our client meetings, he got up and said, “Dave, I would like your guys to do SAN support.”

I was amazed. “Jason, but you’ve always been using NAS, why did you suddenly need a SAN?”
"Well, that's what's the matter," - answered Jason - "We use some application, not that it is even too important for us, but I have to support it, and we have some strange problems with it. I turned to the manufacturer, but they said: “We do not support working with NAS, and we will not help you with setting up our application until you start using SAN.” “I’m even sure,” Jason added, “that the problem is not at NAS, it’s just the guys are looking for a way to get rid of me. I went to our procurement department, with the offer to buy for this task Some small SAN system, but they don’t want to start a new vendor and a new supplier. That's the trouble. They say, “Well, NetApp is a manufacturer of storage, can they not sell us a SAN? If they can't, then find a vendor who can do both NAS and SAN, then we will buy all the storages from them. ”
“So,” added Jason, “you will greatly simplify my life if you can make a SAN stack”

Actually the decision was said, but I did not want to listen to him. It was not what I thought was right, I was sure that the NAS was better. It is cheaper, it works better, and so on, that was my conviction.

NetApp created the NAS market, and the NAS market fueled our success, and we were always its main supporter and promoter. On a more personal level, for many years I was a public "NAS person." At all seminars and conferences, all our competitors promoted SAN, and I went on stage to describe what and why NAS is better. I talked about how low-end technologies set the market in motion, about the shortcomings and doom of the SAN, and how the current situation resembles the story of mainframes that were piled up, ultimately low-end solutions, cheap personal computers. It is difficult to take on the opposite course on the fly and start refuting everything that so often has been publicly proved in words.

Nevertheless, James (James Lau, co-author and co-founder of NetApp) also believed that we should deal with SAN. James is generally the kind of person who always seems to be right, but often keeps quiet about his position. He will express his opinion in order to see how they will react to him, and if they do not heed his opinion, well, he will quietly stay with him, without insisting on his own. Sometimes I even joked that I was his press secretary, because great ideas came to him, and I bring them to the public and “punch through”, repeating to everyone over and over until they listen to them. I knew that our disagreements with James often arise only because one of us knows something that the other does not know. And we sat down to argue.

My arguments were based on the fact that the NAS systems market is growing and growing at a fairly high rate. Many of our employees came to us precisely because they believed in the potential of NAS in the storage market. We positioned ourselves in the eyes of the market and customers as a company engaged in a new direction, rather than traditional solutions. Our sales staff were guided by the “SAN - bad” rule.
In addition, the SAN market was huge and crowded. We would have to come to him as a new and start-up company, without authority and a name in this segment, the most unfortunate situation, and we would have to play “on the siege” of the “big guys” for a very long time. Those resources that we could spend on building our own SAN systems could be more efficiently spent on further improving our already very good NAS systems at that time. Does it make sense to risk our success and the position of the NAS market leader by doing what we thought was worse?

Arguing with me, James argued that the SAN market is huge, and it is significantly larger than the NAS market, while many “big” companies that could be our customers still did not even consider us and our systems seriously because they considered that "NAS is something for home users and small businesses." If we make SAN support, we can at least “get through the door” there. Finally, he thought, most of the work that we have to do and spend the company’s time and money on them will be useful for the NAS.
Since we entered the “big solutions” market, our customers and users have all the time asked us to do one or another opportunity existing in a high-end SAN in our NAS systems, adding the ability to work in a SAN against this background it was not such a big task compared to all the other things that we would have to do in order to satisfy the demands of our customers.
And we argued for quite some time.

Dan, our CEO (Dan Wormenhoven, permanent CEO of NetApp until 2009), wanted to get an opinion of the company's board of directors about this situation with SAN and NAS, so he asked James and me to tell our points of view on the problem. Most CEOs of companies try to protect board members from internal “technical fights” of the company (leaving them with only business), but Dan’s position has always been open and honest with the board, so he was brave enough to invite the two company founders to discuss the issue in their presence .

And I remember Don Valentine (head of Sequoia Capital, the famous investment company Silicon Valley, which initially financed many startups, from Google and Apple to NetApp) , the chairman of our board of directors, asked me “Are your users asking you to make a SAN?”
“Yes,” I replied.
It was shortly after the famous “dot-com crisis”, and Don’s point of view was simple: “You know, in our business, if someone gives you money, I recommend that you take it.”
This clarity of presentation explains why Don Valentine is an excellent chairman of the board of directors.

So James was right, but I was not. "

Soon, starting in 2004, NetApp FAS (Fabric Attached Storage) storage systems were added to work with both NAS (i.e. NFS and CIFS) protocols and SAN, i.e. FC and iSCSI protocols. This became a unique opportunity for data storage systems, at that moment no one offered such a universal system, and this solution became known as Unified Storage, which in Russian would be most accurately translated as “Universal” or “unified storage”.

Depending on whether you need NAS or SAN functionality, you can purchase the necessary license to use the protocol, one or several, which is initially present and supported in the storage system at the level of its internal OS. Thus, if you need, for example, only inexpensive iSCSI, you do not pay when you buy too much, and if you need to use the same storage system with access via any other supported protocol, you do not need to change it to another one, transfer data, retrain personnel , change the monitoring and management software, you simply activate the desired protocol or any other necessary function on the same storage system, and use it further.

The internal architecture of the storage system, with the organization of data storage on top of the special WAFL file structure, which I mentioned earlier, made it possible to organize data storage in LUNs, that is, in block devices used in the SAN, without significant problems.



The “basic configuration” of the dual-controller “cluster” fault-tolerant configuration in the figure is expanded with a dual-port CNA card, but various expansion ports can be installed in this slot besides 10G Ethernet, for example hardware iSCSI HBA, SAS or SCSI ports, or additional FC or Gigabit Ethernet ports.

Such a “modularity” of building a storage system gives extremely broad application flexibility. You can buy a storage system in a “basic” configuration, and then gradually increase its capabilities (and not just the capacity in disks, as usual), as the need arises and the budget is available (and even, in some cases, use the Try & Buy model). Such a design resembles the assembly of a computer when, as necessary (and the emergence of free resources), you add RAM to it, gradually change the processor to a new, more and more powerful, and so on, while your data is on disks, the operating system, and everything else , remains the same, only everything starts to work faster. This is an obvious and significant advantage of such "modular" systems over "monolithic", which, although most often, are initially much faster and more perfect, but can be trivially "not affordable", especially if you simply can’t needed at once.

Where previously we had to use two, or even more different storage systems (separately for NAS, separately for SAN, for which, in turn, for example, separate FC and separate iSCSI), now one could do without, which saved money when buying , operating costs, and even increased disk space utilization, since, for example, available free space remained consolidated and accessible to applications centrally, rather than scattered across several heterogeneous storage systems.

Finally, you could use exactly those data access protocols supported by the applications, approved by its manufacturers, or simply more convenient for you. Since the company's data storage system is usually used as centralized storage for dozens, and sometimes hundreds of servers, for storing data from heterogeneous applications, this is a very important plus.
So by creating Unified Storage, NetApp has simplified life not only for Jason, but for many thousands of system administrators and system architects around the world.