Budget "data center" at Nutanix CE

Those who read our blog for several months already know that a year ago we released a special version of the Nutanix software, Nutanix Community Edition , which, unlike our “adult” systems, sold together with the hardware platform, and costing pretty heavy money, you can download for free, install on your hardware of a suitable configuration, and enjoy. Yes, we don’t have our cool (and even Russian-speaking!) Support (only our English-speaking community-forum, however, is quite active and helpful), but it's free, Karl! And although we initially planned that CE would be used as a learning lab, for experiments, acquaintance with the product, and the like, it is no secret that there are quite a few users who decided to take CE and apply it in production. Well, it is possible, we understand that the times for IT budgets in the country are hard, it would be foolish to refuse such an opportunity.

But the question remains: how and from what to build an “ideal platform” for Nutanix CE, so much so as not to waste money, and get at the output something that will not be terrible to set production tasks. Therefore, in this article I will try to design such a dream-platform under Nutanix CE with a judgment on using it not in the homework lab (I wrote about this version recently in my blog ), but for more or less “combat” tasks.



Well, let's begin. We will build our “combat” platform under Nutanix CE around the motherboard of the well-known “in narrow circles” server assemblers of the SuperMicro company, probably the most eminent and popular vendor of Tier-2 server platform developers.



We, at Nutanix, chose SuperMicro ourselves as a platform provider (it’s no secret that the ready-made Nutanix NX bundles are Supermicro platforms with our specific configuration, with our software on them), and our experience with SM is quite positive. So, we take Supermicro, all the more so that in the huge arsenal of server swag produced by SuperMicro there was a system ideally suited for Nutanix CE, here it is:

Supermicro Mini-ITX 2133/1866 / 1600MHz ECC DDR4 Single socket FCBGA 1667 Motherboard (X10SDV-TLN4F-O)

')





This is a Mini-ITX format motherboard (which is unusual for a server solution), based on an interesting CPU released by Intel last year: Intel Xeon D-1541 .



This is a low-power (45W total) Xeon server processor aimed at the SoC (System-on-Chip) platform. In terms of performance, it roughly corresponds to, for example, the E5-2620v4 , that is, an entry-level server- wide Xeon for single-processor systems.



It also includes two 10G interfaces on our motherboard. The processor has 8 physical cores (16 c HT), 2.1 GHz frequency, 12MB cache and supports up to 128GB ECC RAM .



Of the cost of the motherboard, about $ 600 ($ 581 recommended price Intel), as a result, is just it.



Also, under this mini-ITX, the motherboard SuperMicro produces a nice case, resembling the format of a well and fairly loved by many HP MicroServer, called it: Supermicro Superchassis CSE-721TQ-250B Mini-Tower with 250W Power Supply .







Let's calculate what the platform for Nutanix CE will cost us if we decided to build a platform for serious “combat” services to the company based on this free solution.

I will take Amazon.com prices valid at the time of writing. Sly technology to promote goods on Amazon uses the fact that the product rises in the issuance after the price for it goes down by the seller, which provokes "optimizers" to the constant "swing" with a decrease (and an increase in position in the issue), then return to the previous price higher, and again "discount", with a hit in the list of "best choice". So if you don’t see my price reference link, then most likely this is it. Wait for a few hours, or search for another location for the product name (often from the same seller, under a slightly different name).



So, let's begin. Take to the basket motherboard X10SDV-TLN4F-O

This gives us a motherboard with a processor (it is soldered on the motherboard, as it is SoC), a cooler, network ports, but without memory and disks.



Supermicro Mini-ITX 2133/1866 / 1600MHz ECC DDR4 Single socket FCBGA 1667 Motherboard (X10SDV-TLN4F-O)

$ 899

Source: www.amazon.com/Supermicro-Mini-ITX-1600MHz-Motherboard-X10SDV-TLN4F-O/dp/B010W32T62



Add the CSE-721TQ-250B chassis to it







Supermicro Superchassis CSE-721TQ-250B Mini-Tower with 250W Power Supply

$ 185.99

Source: www.amazon.com/Supermicro-Superchassis-CSE-721TQ-250B-Mini-Tower-Supply/dp/B00REWHHNU



Add memory. This motherboard supports up to 128GB DRAM DDR4 slats of the ECC RDIMM (registered) type and 64GB ECC UDIMM (non-registered). It is a solid capacity for the Mini-ITX server. I remember that in Microserver even with a tambourine more than 32GB did not climb.

We supply 64GB with two strips of 32GB each, with an expansion margin in the future. Such a memory will suit us exactly (Amazon even offers it often with a bundle with our motherboard):







Samsung DDR4-2133 32GB / 4Gx72 ECC / REG CL15 Server Memory M393A4K40BB0-CPB

$ 156.98 x2

Source: www.amazon.com/Samsung-DDR4-2133-Server-Memory-M393A4K40BB0-CPB/dp/B00X04FO9K



Now you need to decide on the SSD and HDD.



Our motherboard has one slot of an increasingly popular M.2 format, I will not dwell on why this is a good choice, in short it is, in fact, a PCIe supporting NVMe . Now Nutanix does not support NVMe as such, only SSD, but we are working in this direction, follow the news.

Since we have this interface, let's not get tired and put the SSD in it.

SSD of this format is much more expensive than all the usual SSD on the SATA interface, but not just like that. Therefore, since we are collecting a dream car, we take M.2 SSD, for example, like this:







Samsung 950 PRO Series - 512GB PCIe NVMe - M.2 Internal SSD (MZ-V5P512BW)

$ 317.99

Source: www.amazon.com/Samsung-950-PRO-Internal-MZ-V5P512BW/dp/B01639694M



However, if you want a cheaper version, then it is possible, for example, with the following SATA SSD:







SanDisk X400 SSD SATA 2.5 "7mm 512G SD8SB8U-512G-1122

$ 135.29

Source: www.amazon.com/SanDisk-X400-SATA-512G-SD8SB8U-512G-1122/dp/B0196FPT1Y



UPD: Discussion of an article in our public group in Telegram gave another option: SATA SSD Intel S3610 at 480GB, for almost the same money as Samsung, but with, roughly, a higher resistance to recordings (by the way, we use it in large "Nutanix NX)



Intel SSDSC2BX480G401 DC S3610 Series 480GB 2.5inch SATA 6Gb / s 7mm MLC SSD Bulk OEM

$ 337.75



If you expect a very high intensity load on the record, then take it.



Source: https://www.amazon.com/Intel-External-Solid-State-SSDSC2BX480G401/dp/B00Q86RDNM



The choice of SATA SSD is now quite wide, choose to your taste, Nutanix does not dictate the choice of model here, except don’t forget that “there is no much SSD capacity” and I recommend that you consider Nutanix SSD capacity from at least 480GB (despite the fact that 200GB is specified in the minimum requirements, part of the SSD is also occupied by the data of the Nutanix system itself, and your data will remain at 200GB quite a bit).



One SSD, especially such a fast one, is still enough for us. Add a couple of HDDs to our server, for example, those that have worked well for the WD Red 2TB:







WD Red 2TB NAS Hard Disk Drive - 5400 RPM Class SATA 6 Gb / s 64MB Cache 3.5 Inch - WD20EFRX

$ 89.99 x2

Source: www.amazon.com/Red-4TB-Hard-Disk-Drive/dp/B00EHBERSE



Please note, we do not take WD Black and RE, simply because in our case nothing depends on the HDD speed. They store cold blocks, so we don’t need high performance SATA HDD, performance will be provided by hot tier SSD, so we won’t be able to load money into expensive HDDs, you won’t get much benefit from this. Otherwise, choose the HDD, which seems to you more correct and reliable, there are no special requirements on the part of Nutanix either.







Almost done. Sum up everything to get the final price:



899 + 186 + 157 * 2 + 318 + 90 * 2 = 1897 USD



You can add 70 dollars for delivery by some mailforwarder (I use the Banderolka for myself, hooking up on their marmalades; D, and I’ve got their point of issue nearby, but there’s a choice on the market, see for yourself).



Total, for ~ 1900 USD (that is, in fact, cheaper, for example, 15 "MacBook Pro!), You get a modern, high-quality, compact, low-power server (8 physical cores, 64GB RAM, 2x10G, SSD), to which You can put our Nutanix CE, and deploy your company virtual machines in it.

In fact, we have now collected an approximate equivalent of the youngest Nutanix model, something like the NX-1065S, a single-socket server with a single SSD and 64GB RAM on an E5-2620v4 CPU. CE will be slower on disk operations, this is due to the fact that we are somewhat differently than in the “big Nutanix” working with disk devices (this was done in order to increase the number of platforms supported by CE), which results on the same platform to the deterioration of disk performance by 30 percent, probably. But this is very good.



What options do we have next?

First, on one server you will not have fault tolerance. For a serious system, this is unacceptable, so we are immediately hammered into a cluster. Nutanix CE supports options from one , three , and four servers in a cluster. Please note that there is no option with two servers, as there is no option with more than four. Two of them do not provide the required fault tolerance and cluster operation logic, and if you need more than four servers in a cluster - welcome to the world of “big guys” :), you have grown into CE.

Servers in a cluster are connected using ethernet-based network interfaces. We have them, I remind you, on our motherboard in abundance. A pair of 10G Base-T and a pair of gigabits. We will use 10G once we have it. But where will we connect them?



For a couple of years now, an interesting product has been present on the market, which dispels the myth that 10G is super-super-expensive, and in general it is for giant enterprises, such as Gazprom, and we are not Gazprom, therefore we will collectively farm LACP on Gigabit RealTec . Down with this here! :)

We dump this into the basket:







NETGEAR ProSAFE 8-Port 10-Gigabit Ethernet Web Managed Switch (XS708E-200NES)

$ 749.99

Source: www.amazon.com/NETGEAR-ProSAFE-10-Gigabit-Ethernet-XS708E-200NES/dp/B01GTWPTJY



Yes, this is it, 8-port (and there are options for more ports: 12, 16 and more) controlled 10G switch.

I will not say that this is the Nexus killer, Arist, and that this is the best switch in the world. Of course not. But he is, he is 10G, he has enough passport bandwith on the switch to work ports without oversubscribing, and it works. If you have money for something better - take it better. But today these netgiry - leaders in Price / Performance. Searching the market you can find options and a bit more expensive (and a little more interesting), from the same Netgear, from Dell Networks, Mellanox. But since we are mastering “budgetary”, take this Netgear.

For a failover switch factory, we take two of them, each of them will have one cable from a pair of 10GBase-T ports of our server, and we still have 4 ports on the switch for some external servers, for example, to enable the cluster to the external network the company.



Next, we can add memory (do not forget that Nutanix CE itself takes approximately 12GB from the host to fit its needs), we have two slots left in each server, we can install two more HDDs to store cold data, or add SSD , or make AllFlash (note that the AllFlash host cannot be mixed in the same cluster with the hosts with the HDD, but you can mix different hosts of the hybrid configuration in the cluster, for example, with different numbers and sizes of disks).



As a result, for an amount of less than 10K USD, we obtained a highly available hyperconvergent virtual data center from four server virtualization hosts of a completely non-weak configuration, 10G network backbone, with excellent expansion prospects (if the need arises), and a completely free software hypervisor part. And all this in the amount of 19 "rack total of about 8U .



In my opinion, a very impressive result.



UPD: Acropolis Block Service and Docker Support are now included in Nutanix CE version 2016.06.30, which we wrote about in the post about version 4.7.