How to build the infrastructure of the building. class c using servers Dell R730xd E5-2650 v4 worth 9000 euros for a penny?
In this article, we will describe what infrastructure solutions our customers implement using Dell R730xd server platforms and why the price for renting this platform in a TierIII + level European data center with excellent communication channels to Ukraine and Russia, as well as in 9 locations in the US , already with accommodation and connectivity priced at $ 249 / month for 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480 SSD 1Gbps has become a reality. We will share the possible solutions on the basis of these platforms using private vlan, 10G local network and CISCO hardware Firewall, which are available to our customers upon request. And also, in the best traditions, we will offer a bonus in the form of a free period of use of Dell R730xd servers for Habrahabr readers.
Recently, we have received more and more requests for the construction of various corporate infrastructures, and this, unfortunately, is connected not so much with the fact that we provide solutions of very high quality, but with the price of these solutions and the level of security and rule of law which is provided in the Netherlands and the USA, but alas, often unavailable in Ukraine and Russia. Where, unfortunately, such decisions are simply "space" money, since the concept of "long money" is alien to post-Soviet countries, in principle, against the background of other risks or in all the lack of the necessary infrastructure and level of certification.
')
A good example is the difference in the price of iron. For example, the Dell R730xd platform, purchased for our customers, in the base configuration 2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD costs about 9000 euros . Of course, there is no question of any low rental price for this server in Ukraine or Russia, since the rental cost is based on the payback period of the platform for 12-18 months. This means that the minimum possible rental price, excluding the cost of accommodation, electricity and communication channels - will be in the range of $ 500-800 / month, depending on the risk level of the provider and the business plan. And you need to find a good data center with the required level of certification and communication channels. And do not forget about taking into account the risks of illegal seizure of equipment in the course of any illegal investigative actions that may arise in the form of competition and the specifics of doing business in a particular region.
Therefore, it is not surprising that our proposal has aroused great interest, especially since the data centers in which we provide the proposed servers have many necessary and important for the corporate sector certificates - ISO 27001 , PCI DSS , SOC 1 , HIPAA and NEN 7510 .
Now we will look at several cases that may be interesting, both for beginners and small projects, and for fairly large system integrators who build solutions in their offices in germozonah or in data centers.
It is no secret that the requirements for data storage and processing are constantly growing, and at an accelerated pace. If a few years ago it was enough to have a storage in 1TB, which provided several hundred IOPS, now the needs have increased to tens of thousands of IOPS and petabyte space. These capacity and performance requirements are fueled in part by an increase in the amount of unstructured data, including music, images, video, database backups, log files and other archives, financial and medical data, that is, data known as “Big Data”. Not to mention the growing requirements for data storage as a result of the spread and expansion of the Internet network and new Internet resources. Thus, with all these power requirements, customers' expectations regarding high reliability and high performance are greater than ever.
While IT companies are trying to solve the problems of managing petabytes and even exabytes of data, the cloud storage model is becoming more common in modern data centers. More and more new software packages are being written that allow you to tune the interaction of the cloud environment with iron in an optimal way, Ceph is one such development.
Ceph is an open source distributed storage system designed to provide high performance, reliability and scalability. Ceph implements object storage on a distributed computer cluster and provides interfaces for storing objects, blocks, and files. Ceph provides fully distributed storage without a single point of failure and scalable to petabyte level. Ceph replicates data and thus provides fault tolerance. The system is designed in such a way as to not only ensure self-recovery, but also control, which helps to avoid unnecessary maintenance costs. Since Ceph uses universal hardware controlled by software, and management functions are available through application programming interfaces (APIs), it is referred to as software-defined repositories (SDS).
Red Hat Ceph Storage is a ready-to-implement storage, software-defined storage that is open, adaptable, scalable, and widely supported. Combines innovation from open-source development with technical core and support from Red Hat. The solution provides tight integration with OpenStack and is designed from the ground up to provide next-generation storage for both cloud environments and other workloads.
Here we will look at an example of implementing this solution on Dell servers, in particular the Dell PowerEdge R730xd, which we offer for rent, we will consider the advantages of building storage based on these platforms. Perhaps this information will be useful to architects, engineers, and IT administrators who wish to explore the benefits of using Red Hat Ceph Storage on Dell PowerEdge servers and who need to design and plan implementations using proven best practices.
But at the beginning:
Dell PowerEdge R730xd is certainly one of the best platforms for corporate sector tasks and not only, it has won many different awards and is quite an effective solution for organizing data warehouses, as it can provide high density data storage for reasonable money.
The PowerEdge R730xd is available in three chassis configurations with various form factors and drive combinations, as well as an optional rear expansion unit:
- 24 available front 2.5-inch drive bays for SAS, SATA or nearline SAS plus 2 optional 2.5-inch drive bays at the back. The 2.5-inch chassis can optionally support up to 4 PCIe Express Flash drives from Dell in the front end.
- 12 available front 3.5-inch bays for SAS, SATA or nearline SAS drives with 4 optical internal 3.5-inch bays, hot-swappable drives, plus two optional 2.5-inch bays at the back.
- 18 front-sided 1.8-inch SATA bays, 8 3.5-inch bays for SAS, SATA or nearline SAS drives, plus 2 optional 2.5-inch bays at the back.
It would seem that every chassis is good for a kind of tasks. That is certain. But are they equally cost effective?
We opted for the second option to meet the needs of the widest possible range of users on a chassis with 12 drive bays. Since we consider it the most effective. And that's why. The economic efficiency of the solution is already manifested in its versatility - you can put drives of various form factors into this chassis and it is cheaper to purchase, and also, which is important, you can get more performance when working with data using several platforms connected to each other than one with high storage density and obviously lower productivity.
We believe that connecting several similar platforms to a multi-gigabit local network (it is possible to connect each node to a local network at speeds of 20 Gbit / s and more, using two-port Intel X540-T2 ten-gigabit cards, which we provide additionally) can achieve much better results and performance than platforms with a higher density of drives. What is especially critical for infrastructures that use these platforms for working with databases. Alas, but with a large number of drives, the controller can be quite overloaded, and an additional controller in version xd, alas, is not available. The performance that can be achieved with the use of platforms with 12 compartments and a local multi-gigabit network will be much higher, and the solution in turn will be more distributed and reliable. In a word - cost-effective!
The Dell PowerEdge R730xd platform, which is only 2U in size, supports up to 2 Intel Xeon E5-2600 v3 processors and higher, that is, it allows you to have up to 36 cores when using 18-core processors. We chose an intermediate version, but the latest generation - a 12-core E5-2650 v4 processor (24 cores at your disposal in total, and 48 threads in the light of multithreading), since it turned out to be the most cost-effective. So in the fourth generation implemented more efficient processor instructions, for example, AES, responsible for data encryption, 70% more efficient than the same processors, but the third generation. In this case, the processor and the platform can support up to 1.54 TB of RAM, which is important in some very specific cases. We chose the most mass version, the most affordable one and speed of work - 128GB DDR4 RAM and provided for the possibility of an upgrade upon the request of subscribers.
On the front panel of the R730xd, there are 6 system status indicators that can report various problems, thereby avoiding a lot of critical problems by taking appropriate measures in a timely manner. DIMM slots for RAM are located directly on the motherboard. R730xd supports register-based DIMM with error correction, as well as LRDIMM (Load Reduced Dual In-Line Memory Modules), or the so-called module with reduced load, is a relatively new type of memory for servers. But we do not use it, since LRDIMMs become useful only with large amounts of memory, when there is a goal to increase the speed of work.
The Dual-SD Internal Module (IDSDM) allows customers to deploy Dell resilient virtualization technology, which provides failover redundancy for embedded hypervisors. Despite the fact that PowerEdge R730 supports GPUs, which can be very useful in cases of building virtual office infrastructure (virtual desktops), as well as for high-performance computing and co-processing, R730xd does not support graphics processors, because proper cooling cannot be provided video card. However, so far we have a small demand for this service and the video card in the server has been ordered only by one of our subscribers. It is for this reason that we did not order the R730 platform en masse, but we can deliver them on request, having equipped one of the recommended cards.
Due to this, unfortunately, the price for a rental solution can not be so attractive and increases by more than 2 times, depending on the period of payment and the term of the contract. We offer the cards themselves to be purchased through our supplier in the Netherlands, prices are given below and only from this recommended list (such were the recommendations of Dell itself when we made the request), perhaps this information will be useful to someone:
NVIDIA Tesla M10 CusKit GPU: 2,884.98 EUR
NVIDIA Tesla M40 GPU: 4,913.33 EUR
NVIDIA Tesla M40 24GB GPU, Cust Kit: 6,458.95 EUR
NVIDIA M60 GPU, Passive, Requires GRID 2.0 SW for VDI Function, Cust Kit: 5,094.95 EUR
Licenses:
Nvidia GRID vApps Subscription License 3 yr, 1 CCU: 20 EUR
Nvidia GRID vPC Subscription License 3 yr, 1 CCU: 95 EUR
Nvidia GRID vWS Subscription License 3 yr, 1 CCU: 480 EUR
So, if you are ready for at least a 2-year contract to rent a Dell R730 server (not xd, because the solution is much more expensive) - contact sales@ua-hosting.company, we will be happy to help you! Using one of the recommended video cards in the configuration DELL R730 2 x E5-2650 v4 / 128GB / 6 x 480GB SSD / 1Gbps 100TB + GPU and a 2-year contract, it will cost approximately $ 6,816 per year, instead of $ 2,988 per year in the case The Dell R730xd + platform article will require payment of the cost of a video card and license, as this is very specific equipment.
Nevertheless, even this price is much more attractive than the prices that data centers in Ukraine and Russia can offer for such solutions, if they ever want to provide such solutions ... For, oddly enough, in the Netherlands, where cooperation with Dell is debugged directly, the promised the delivery time of the server with the video card was almost 2 months from the time the order was placed by our client (the client agreed to expect, since there are simply no alternatives on the market) , due to the strong specificity of this product, it was simply not available from Dell itself . Nevertheless, the delivery was made on the basis of faster than in a month. Thanks to the data center supply department and the guys from Dell for their promptness. But for me, yet this experience has become unusual, since the standard Dell R730xd platforms are shipped within a few days.
Server management can be accomplished through a convenient iDRAC8 (Integrated Dell Remote Access Controller 8) with a Dell life cycle controller, which reduces the time spent on management tasks, reduces the likelihood of errors, improves security and improves the overall efficiency of your IT environment.
For our subscribers, access to the iDRAC8 is provided via RMI (Remote Management Interface), which is available exclusively from the private data center network for security purposes, access to which we provide through the Open VPN tunnel for free. After logging in, the iDRAC displays an overview of the system, as well as viewing the virtual console via iKVM.
I would like to pay special attention to monitoring. IDRAC8 allows you to get power consumption statistics for the last hour, day or week and limit the maximum power consumption:
The iDRAC temperature monitoring and recording system allows you to collect data from temperature sensors and determine the range in which it is located. Thus, it is considered acceptable to stay processors in the Warning temperature range only for 10% of the time during the year, and critical - for 1%. Time in the critical band also affects the allowable time spent in the warning band. The collection of temperature data begins when the system is turned on after being shipped from the factory and cannot be reset.
Dell's 13th generation PowerEdge servers are equipped with PERC9 controllers, in which the LSI CacheCade technology previously used in PERC8 controllers has been replaced with a more efficient one — SanDisk DAS Cache.
In the course of synthetic tests, DAS Cache gives a significant acceleration of the performance of HDD arrays in cases of hybrid configurations of HDD + SSD. So, in the case of an array of 5 HDDs in RAID6 (the 6th level is selected for maximum capacity) and 5 SSDs RAID10 (4 + 1 hot spare SSDs for maximizing the performance of the array) using DAS Cache, the performance of the array is 5 HDD RAID6 + DAS Cache on the specified SSD array turned out to be close to the performance of the SSD array itself:
So, having a little familiarized with the platform, now we can consider the advantages of this platform when building storages.
Red Hat Ceph Storage uses industry standard servers for scalability, resiliency and performance. Data protection techniques play a key role in solution costs. Ceph allows the user to select different protection methods for different storage pools.
Replicated storage pools produce complete copies of stored objects and are ideal for quick recovery and data handling. In the case of a replicated storage pool, Ceph defaults to a replication ratio of three when three copies of data reside on three separate Ceph nodes.
Storage pools with protection against data corruption represent a single copy of data with parity, which is useful in case of need for durable data storage and is cost-effective.
The PowerEdge R730xd is an extremely flexible and scalable dual-bay rack server that provides high-performance processing and a wide range of local storage capabilities with workload optimization, including the ability to use hybrid levels. The development includes a wide range of configurations, which is ideal for Ceph.
- R730xd provides the ability to create highly efficient infrastructure solutions for both small and large businesses.
- R730xd can reduce the time of commissioning due to the possibilities of automated deployment, which reduces the number of actions by the user and reduces the number of human errors.
- PowerEdge servers improve the performance of IT infrastructure in data centers using innovative management tools such as iDRAC Quick Sync and iDRAC Direct, which provides the ability to monitor the state of the system comprehensively and perform rapid deployment.
- In PowerEdge servers, energy usage is optimized, which provides better performance per Watt of power consumption, and also allows you to more accurately control energy consumption and cooling.
The Dell PowerEdge R730xd delivers the benefits of maximum performance by:
- Accelerate application efficiency by using the latest technology and dynamic local storage.
- Scaling is easy and fast using front-mounted storage, which supports drives from a wide range - from low-cost SATA HDDs to ultra-fast 2.5 "SSDs, as well as very low latency drives, such as PowerEdge Express Flash NVMe PCIe SSDs.
- R730xd is well adapted for building hybrid-type storages thanks to the use of new caching technologies in the PERC controller, which increases the economic efficiency of the Ceph solution.
An example of infrastructure implementation with a Red Hat Ceph Storage storage, which consists of 5 Dell R730xd servers, you can see below:
The following independent results were obtained during independent tests of various configurations of the Dell R730xd platforms:
Our version of the platform shows a very good result, and in the case of reading, perhaps the best. And if we take into account the fact that we ordered these platforms in large quantities and due to this we can offer a price more than 2 times lower than on the other platform options - it turns out to be the best consolidated for reading and writing on the cost-effective parameter. In the end, nothing prevents to build a cluster of 7-8 platforms, instead of 5. At the same time, to get even greater fault tolerance.
Are 10 Gigabit Switches Required When Building Such Infrastructures? No, not mandatory, one of our clients built the infrastructure, though from 3 servers, using the following local area network connection:
That allowed him to save not a lot of money - $ 350 / month for renting a switch, since 10-Gigabit switches are quite expensive, and we provide only the best - Arista 7050tx-48-r. In decisions on several ports we see no point.
Not so long ago, Microsoft conducted an interesting test of the performance of these platforms in the case of building data warehouses with a small number of nodes.
Four Dell R730xd nodes were used, combined into a local 100 Gigabit network using the 32-port Arista DCS-7060CX-32S 100Gb switch, running EOS version 4.15.3FX-7060X.1.
The configuration of the nodes used is as follows:
2x Xeon E5-2660v3 2.6Ghz (10c20t)
256GB DRAM (16x 16GB DDR4 2133 MHz DIMM)
4x Samsung PM1725 3.2TB NVME SSD (PCIe 3.0 x8 AIC)
Dell hba330
4x Intel S3710 800GB SATA SSD
12x Seagate 4TB Enterprise Capacity 3.5 ”SATA HDD
2x Mellanox ConnectX-4 100Gb (Dual Port 100Gb PCIe 3.0 x16)
Mellanox FW v. 12.14.2036
Mellanox ConnectX-4 Driver v. 1.35.14894
Device PSID MT_2150110033
Single port connected / adapter
With VMFleet, 20 virtual machines were raised on each node, that is, only 80 virtual machines. Each virtual machine was configured with 1vCPU. Then VMFleet was used to run the DISKSPD utility to test storage performance on each of the 80 nodes, the utility is available here for free. Test parameters are as follows: 1 stream, 512KiB sequential reads for 4 I / O operations.
As you can see in the image above, it turned out to achieve an incredible result of 60 gigabytes per second in total, which is approximately equal to 5 English versions of Wikipedia in compressed form (11.5GiB), which are downloaded every second. And the speed from each virtual machine was about one CD per second - 750 MB.
This test perfectly demonstrates how effective a solution can be when the three components — computing power, storage, and network — are balanced, which minimizes the potential bottlenecks that can occur in an unbalanced system.
What about platforms from other manufacturers?
For comparison, the performance was chosen iron weaker than is offered for rent by us - servers with an E5-2620v3 processor, and not E5-2650v4, which are much more productive. The main testing methodology used for this comparison was to measure the amount of IOPS. Several different workload simulations were used, namely, all Read and 30% Read / 70% Write (an analogue of an OLTP system, a transactional system, when processing goes in small transactions, but with a large flow, and customers need to ensure a minimum response time).
Tests were conducted several times to eliminate the negative impact of hidden factors on system performance. First, the base tests were carried out on the Dell R730xd and HP ProLiant DL380 using only HDD drives in the storage (5x1TB HDD RAID5), which served as a benchmark for performance. Then, the same test suite was launched with 5x1TB HDD RAID-5 storage using DAS cache installed on SSDs with SAS interface (2xSAS 480GB Samsung SSD RAID1) on the Dell server, and on the HP server using HPE SmartCache with similar set of drives for storage and caching. Finally, the latest test suite was launched on a Dell server using DAS Cache on NVMe PCIe SSDs (2x400GB Samsung NVMe in Software RAID) to display the maximum contribution of caching to application performance gains in cases of intensive read and write. A similar test for the HP server was not performed, because Smart Cache does not support NVMe disks for caching.
System configuration:
Several random read / write tests were conducted with different block sizes - 4 and 8KB. 100% , 70% 30% . , , 900 (15 ), , .
IOPS ( / ):
% :
Dell R730xd SanDisk DAS cache HPE SmartCache, NVMe PCIe SSD . OLTP R730xd IOPS 3- SanDisk DAS Cache SAS SSD 6 SanDisk DAS Cache + NVMe PCIe SSDs HPE's SmartCache. SanDisk DAS Cache, SanDisk DAS Cache + NVMe PCIe SSDs IOPS .
, HP 6% IOPS 41 , 1 IOPS 0,15% , Dell R730xd 16% 264 , — 0,06% 1000 IOPS.
, , Dell ( 2,5 ), % , , — , .
Dell R730xd HP ProLiant DL380 ( ), , cost-effective.
, , — , , , .
. , , , , . , , 5-2650 v4, , 70% ( AES), 5-2650 v3.
«» low-ost, ? ? ? — . , . - , latency , , latency !
, 10 , 9 , , . 5-2640 v4, 10, 12 , .
https://browser.primatelabs.com/v4/cpu/768278 — Geekbanch 3 Dell R730xd c 5-2640 v4
, , , , SanDisk DAS cache, , , ! - .
. AES, 1- 1000 , , , 100 .
, , , , .
, , , .
VMware Bare Metal ( ) Dell R730xd . , .
? - . , , - ( ) — . , , , , , … , , , , , , , , , 5 - … , , « » - - .
, . , :
, 15-30 .
100%, , 100%. 99,9% , . 0,1% — 44 , — . 99,99% , 4 . 2 , , . , 40 , . . 5 - , . blackout - — - , - , - . , , , , , , 3 , , 3 . 30 3 ? , .
, ? , . , : , - . , 40 , , — . 40 , 40 — , . , .
, , , .
. bare metal Dell , , , , , . , cost-effective .
. — , . , , , 3 , , . Dell R730xd , . , VMware.
Many of our customers who use these servers for databases have really liked this platform. And not only because, unlike the Dell R730, the xd version can be provided with support for NVMe PCIe SSDs, which provide minimal latency, which is extremely important for such solutions. But also because the controller is incredibly effective at work even with the usual 2.5 "SSD, which we offer to customers.
Of course, there is a drawback to the Dell R730xd - the controller is only 1, while in the Dell R730 there are 2 hardware controllers for 26 drives. But this is not a significant problem, since we chose a platform with 12 compartments for storage and therefore you will not get a bottleneck in the form of insufficient controller performance when working with this platform and a large number of SSDs. And the advantage in the form of the possibility of installing the NVMe PCIe SSD fully covers this disadvantage.
As mentioned above, not everyone needs complex database clusters, the performance that this platform can provide is much more important. One of our clients did just that, abandoned the excesses of virtualization and building complex clusters, renting a VPS with dedicated drives for database backups from the main Dell R730xd server: VPS (KVM) - E5-2650 v4 (24 Cores) / 40GB DDR4 / 4x240GB RAID10 SSD 1Gbps 40TB - $ 99. Of course, in case of a problem with the main node, there will be a slight decrease in performance and the need for manual switching will arise, however, this is more than a cost-effective solution, given the limited budget.
We do not call everyone for such savings, in some cases it is even very justified. However, by increasing the budget by only $ 230 per month, you can allow the E5-2650 v4 node that is already identical and connect them together to a 20-gigabit local area network, which in principle allows you to build a small cluster with acceptable latency even without using PCIe NVMe SSDs.
The solution itself will look like this (with a basic set of drives):
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $ 289 / month
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $ 289 / month
Increasing their number to increase performance does not always make sense. So, during tests with databases, we found that a solution with 8 SSDs is more efficient than a solution with 4 SSDs by only 35%.
Regarding the recommended RAID levels, in some cases RAID5 can be very optimal. Of course, RAID10 provides better performance, however, what if the drives are so fast that using them in RAID10 can lead to a controller becoming a bottleneck? In such cases, building a HW RAID RAID5 array is much more profitable using hot-spare drives than RAID10 (if the goal is not to lose half the SSD quota).
Do I need the controller cache to be on or better off? The RAID controller has a very limited amount of memory. What is 1GB in case of working with SSD? In cases with fast SSDs, there is no sense in the included cache for reading, since filling memory with read operations that will not become faster at all, since SSDs are fast enough anyway, we don’t leave enough cache for writing, and, as we know, in inexpensive SSDs - the bottleneck is the write operation, due to a drop in performance with insufficient over-provisioning and 100% utilization of the free quota (perhaps a drop tenfold). Therefore, we recommend leaving only the write cache enabled. Thereby ensuring maximum effectiveness of this solution.
Is NTFS block size so important? For most storage, 64KB is listed as the optimal block size for maximum performance. But very often we are faced with cases where the client is already using the server for the databases, applying the initial configuration with the standard NTFS cluster size of 4K. To fix this, you need to reformat the drives, but is it really a big difference in the general case? Do I need to expose myself to the possible downtime and inconvenience associated with migration?
The controller in the Dell R730xd server provides so good performance even with 4 SSD drives and a write cache enabled (a very important point, the read cache must be disabled because it does not provide any benefits, but only reduces the efficiency of write operations due to the lack of space for full cache, as indicated above), even with a standard block size of 4K, we get very good performance indicators, which are extremely good.
Nevertheless, we always go to a meeting and are ready to offer a similar platform for a period of time for work by subscribers, if something was not taken into account by them.
RAMdrive still turns out to be faster than SSD drives. If your requests can be processed in TempDB, for example, sorting or merging, then using RAMdisk (a program that turns part of your RAM into a drive) can be quite justified. So, comparing the speed of 8 SSD drives in RAID10 with RAMdrive, we found that it is about 4 times faster with 4K write operations with queue depth of 32, which simulates the operation of TempDB files. Even combining 8 SSDs in RAID0 failed to beat the results of RAMdrive.
RAID5 in the case of 8 SSDs may be more efficient than RAID10. 4K read operations with a queue depth of 32% faster by about 40%, and this is not surprising, given the features of RAID5. RAID5 wins in performance in other tests, so with sequential writing it is 20%, is it inferior only with 4K write operations with a large queue depth (in our case 32) and turns out to be less effective by 30%. So the use of RAID5 can in some cases be very cost-effective.
More storage does not always mean faster. When we deal with random small operations, adding drives to an array after a certain limit does not improve performance. It would seem why? It turns out everything depends on the capabilities of the controller, which is forced to record data on a whole bunch of drives. That is why we purchased platforms with a maximum of 12 drives in order to reduce the likelihood of a bottleneck. However, you need to look at the intensity of operations within each project, sometimes 4 larger drives, say 960GB, can be a better and more cost-effective solution than a bunch of 480GB drives.
Aerospike is an open source NoSQL database optimized for flash memory, providing high speed and scalability. Aerospike is designed to provide very low latency for read requests and a large write load, which satisfies the current business requirements of those who need to handle a large number of I / O operations. Aerospike directly manages local storage on the nodes of its cluster, regardless of the source file systems, be it DRAM, flash or traditional hard drives. This allows you to store critical data on the fastest media, the fastest processing of which is most critical, while the distributed nature of the Aerospike database provides a high degree of reliability in the event of a cluster node failure.
The 13th generation Dell servers, namely the PowerEdge R730xd, with the latest E5-2650 v4 processors and fast DDR4 SDRAM, provide incredible computing power. And the ability to use NVMe SSDs, such as the Samsung SM1715, can help get a lot of performance in nodes with high loads with minimal latency. That allows you to meet modern SLA requirements for the processing of a single transaction with a reliable level of fraud protection even in very large databases, making the decision acceptable even for the financial sector and bank transfers, where transactions are sometimes carried out within a few hours or even days.
For the banking sector, we can offer a solution based on these platforms with a local network of 40 and 100 Gbit / s between the cluster nodes and the use of appropriate switches.
We are pleased to offer our subscribers the highest security standards. The data centers in which we offer these servers have many certifications: ISO 27001 , PCI DSS , SOC 1 , HIPAA and NEN 7510 .
But in order to meet the requirements of the same Google, some clients need to build so-called demilitarized networks (DMZ networks) - networks with enhanced security, which implies a hardware firewall that restricts access to equipment and allows you to hide the necessary components behind the firewall and increase safety more.
And everything is ok, when such a solution is ordered immediately, but it is very difficult to implement a transition almost without downtime in the process that recently happened to one of our clients, with a project to process applications for visas for citizens from different countries of the world, where it is not available even during hours can result in losses of tens of thousands of dollars. But we found a solution, went to the meeting, put the additional equipment in the closet, moving less important, and included the firewall according to the scheme, as the client wished, so that he could migrate without downtime.
It was / It became
Perhaps this scheme will also be useful for someone to carry out such a migration, as engineers in data centers do not always want to see obvious things, but insist on downtime at 2 o'clock to switch to the firewall, but I beg you - order a hardware firewall all well in advance if you need it. We are flexible, and always go to a meeting, but in some situations our opportunities may be limited.
In future publications, we will try to conduct a series of additional tests and share additional experience related to the use of these platforms, which will further enhance the understanding of the compilation of the most effective solution. After all, it is surprising to see the fact that RAID0 is from two SSD drives, in cases of high load databases, it may turn out to be slower than a single write drive, that is why we receive requests for NVMe PCIe SSD from time to time. It is likely that we will show the most effective scenarios for implementing solutions, share additional experience when building multiple RAID arrays within a single RAID controller turns out to be effective, show in which cases two independent RAID10 arrays from 4 drives will be better than one RAID10 array out of 8. When will it be efficient to use RAID1 for TempDB, and RAID5 for everything else. And how can you do without NVMe PCIe SSD, when there is a limited budget.
We want to provide you with the best and have done everything for this by offering unique prices for the Dell R730xd:
ua-hosting.company/serversnl - in the Netherlands
ua-hosting.company/serversus - and in the USA
Configurations can be very diverse, for example, the following are available for instant activation in the Netherlands :
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12x240 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 4x4TB 4x480 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12x4TB SATA 1Gbps 100 TV - * $ 249
And honest dedicated 1Gbps Unmetered (without traffic) is available to them for + $ 120 / month. Plus the ability to purchase a firewall, network cards, a switch and build a multi-gigabit local area network. But the condition of the price is valid for a year contract.
We want to improve the condition and offer a bonus for short-term payment and rent without a contract, making the offer more affordable. For anyone who orders and pays for any of the proposed options in the Netherlands for a period of 1 month , we will provide the exact same server for 1 month completely free of charge , or we will recalculate the ordered configuration and calculate the ordered server at a price as with an annual contract ($ 249 / month instead of $ 369 / month), returning the difference to the balance. To do this, simply leave the order number in the comments. For those who have not yet reached dedicated solutions, we suggest renting any VPS (KVM) with dedicated drives in the Netherlands or the USA on these platforms, for example, VPS (KVM) - E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps 10TB - $ 29 , and get a bonus of 1-4 months of use when paying for 1, 3, 6, 12 months, respectively, with the number of the order in the comments. Now build infrastructure has become easier!
Recently, we have received more and more requests for the construction of various corporate infrastructures, and this, unfortunately, is connected not so much with the fact that we provide solutions of very high quality, but with the price of these solutions and the level of security and rule of law which is provided in the Netherlands and the USA, but alas, often unavailable in Ukraine and Russia. Where, unfortunately, such decisions are simply "space" money, since the concept of "long money" is alien to post-Soviet countries, in principle, against the background of other risks or in all the lack of the necessary infrastructure and level of certification.
')
A good example is the difference in the price of iron. For example, the Dell R730xd platform, purchased for our customers, in the base configuration 2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD costs about 9000 euros . Of course, there is no question of any low rental price for this server in Ukraine or Russia, since the rental cost is based on the payback period of the platform for 12-18 months. This means that the minimum possible rental price, excluding the cost of accommodation, electricity and communication channels - will be in the range of $ 500-800 / month, depending on the risk level of the provider and the business plan. And you need to find a good data center with the required level of certification and communication channels. And do not forget about taking into account the risks of illegal seizure of equipment in the course of any illegal investigative actions that may arise in the form of competition and the specifics of doing business in a particular region.
Therefore, it is not surprising that our proposal has aroused great interest, especially since the data centers in which we provide the proposed servers have many necessary and important for the corporate sector certificates - ISO 27001 , PCI DSS , SOC 1 , HIPAA and NEN 7510 .
Now we will look at several cases that may be interesting, both for beginners and small projects, and for fairly large system integrators who build solutions in their offices in germozonah or in data centers.
The Benefits of Dell R730xd Platforms for Building Data Warehouses Using Red Hat Ceph
It is no secret that the requirements for data storage and processing are constantly growing, and at an accelerated pace. If a few years ago it was enough to have a storage in 1TB, which provided several hundred IOPS, now the needs have increased to tens of thousands of IOPS and petabyte space. These capacity and performance requirements are fueled in part by an increase in the amount of unstructured data, including music, images, video, database backups, log files and other archives, financial and medical data, that is, data known as “Big Data”. Not to mention the growing requirements for data storage as a result of the spread and expansion of the Internet network and new Internet resources. Thus, with all these power requirements, customers' expectations regarding high reliability and high performance are greater than ever.
While IT companies are trying to solve the problems of managing petabytes and even exabytes of data, the cloud storage model is becoming more common in modern data centers. More and more new software packages are being written that allow you to tune the interaction of the cloud environment with iron in an optimal way, Ceph is one such development.
Ceph is an open source distributed storage system designed to provide high performance, reliability and scalability. Ceph implements object storage on a distributed computer cluster and provides interfaces for storing objects, blocks, and files. Ceph provides fully distributed storage without a single point of failure and scalable to petabyte level. Ceph replicates data and thus provides fault tolerance. The system is designed in such a way as to not only ensure self-recovery, but also control, which helps to avoid unnecessary maintenance costs. Since Ceph uses universal hardware controlled by software, and management functions are available through application programming interfaces (APIs), it is referred to as software-defined repositories (SDS).
Red Hat Ceph Storage is a ready-to-implement storage, software-defined storage that is open, adaptable, scalable, and widely supported. Combines innovation from open-source development with technical core and support from Red Hat. The solution provides tight integration with OpenStack and is designed from the ground up to provide next-generation storage for both cloud environments and other workloads.
Here we will look at an example of implementing this solution on Dell servers, in particular the Dell PowerEdge R730xd, which we offer for rent, we will consider the advantages of building storage based on these platforms. Perhaps this information will be useful to architects, engineers, and IT administrators who wish to explore the benefits of using Red Hat Ceph Storage on Dell PowerEdge servers and who need to design and plan implementations using proven best practices.
But at the beginning:
A little about the platform itself, what option do we offer and why is it cost-effective?
Dell PowerEdge R730xd is certainly one of the best platforms for corporate sector tasks and not only, it has won many different awards and is quite an effective solution for organizing data warehouses, as it can provide high density data storage for reasonable money.
The PowerEdge R730xd is available in three chassis configurations with various form factors and drive combinations, as well as an optional rear expansion unit:
- 24 available front 2.5-inch drive bays for SAS, SATA or nearline SAS plus 2 optional 2.5-inch drive bays at the back. The 2.5-inch chassis can optionally support up to 4 PCIe Express Flash drives from Dell in the front end.
- 12 available front 3.5-inch bays for SAS, SATA or nearline SAS drives with 4 optical internal 3.5-inch bays, hot-swappable drives, plus two optional 2.5-inch bays at the back.
- 18 front-sided 1.8-inch SATA bays, 8 3.5-inch bays for SAS, SATA or nearline SAS drives, plus 2 optional 2.5-inch bays at the back.
It would seem that every chassis is good for a kind of tasks. That is certain. But are they equally cost effective?
We opted for the second option to meet the needs of the widest possible range of users on a chassis with 12 drive bays. Since we consider it the most effective. And that's why. The economic efficiency of the solution is already manifested in its versatility - you can put drives of various form factors into this chassis and it is cheaper to purchase, and also, which is important, you can get more performance when working with data using several platforms connected to each other than one with high storage density and obviously lower productivity.
We believe that connecting several similar platforms to a multi-gigabit local network (it is possible to connect each node to a local network at speeds of 20 Gbit / s and more, using two-port Intel X540-T2 ten-gigabit cards, which we provide additionally) can achieve much better results and performance than platforms with a higher density of drives. What is especially critical for infrastructures that use these platforms for working with databases. Alas, but with a large number of drives, the controller can be quite overloaded, and an additional controller in version xd, alas, is not available. The performance that can be achieved with the use of platforms with 12 compartments and a local multi-gigabit network will be much higher, and the solution in turn will be more distributed and reliable. In a word - cost-effective!
Features of the structure and design, the use of video cards
The Dell PowerEdge R730xd platform, which is only 2U in size, supports up to 2 Intel Xeon E5-2600 v3 processors and higher, that is, it allows you to have up to 36 cores when using 18-core processors. We chose an intermediate version, but the latest generation - a 12-core E5-2650 v4 processor (24 cores at your disposal in total, and 48 threads in the light of multithreading), since it turned out to be the most cost-effective. So in the fourth generation implemented more efficient processor instructions, for example, AES, responsible for data encryption, 70% more efficient than the same processors, but the third generation. In this case, the processor and the platform can support up to 1.54 TB of RAM, which is important in some very specific cases. We chose the most mass version, the most affordable one and speed of work - 128GB DDR4 RAM and provided for the possibility of an upgrade upon the request of subscribers.
On the front panel of the R730xd, there are 6 system status indicators that can report various problems, thereby avoiding a lot of critical problems by taking appropriate measures in a timely manner. DIMM slots for RAM are located directly on the motherboard. R730xd supports register-based DIMM with error correction, as well as LRDIMM (Load Reduced Dual In-Line Memory Modules), or the so-called module with reduced load, is a relatively new type of memory for servers. But we do not use it, since LRDIMMs become useful only with large amounts of memory, when there is a goal to increase the speed of work.
The Dual-SD Internal Module (IDSDM) allows customers to deploy Dell resilient virtualization technology, which provides failover redundancy for embedded hypervisors. Despite the fact that PowerEdge R730 supports GPUs, which can be very useful in cases of building virtual office infrastructure (virtual desktops), as well as for high-performance computing and co-processing, R730xd does not support graphics processors, because proper cooling cannot be provided video card. However, so far we have a small demand for this service and the video card in the server has been ordered only by one of our subscribers. It is for this reason that we did not order the R730 platform en masse, but we can deliver them on request, having equipped one of the recommended cards.
Due to this, unfortunately, the price for a rental solution can not be so attractive and increases by more than 2 times, depending on the period of payment and the term of the contract. We offer the cards themselves to be purchased through our supplier in the Netherlands, prices are given below and only from this recommended list (such were the recommendations of Dell itself when we made the request), perhaps this information will be useful to someone:
NVIDIA Tesla M10 CusKit GPU: 2,884.98 EUR
NVIDIA Tesla M40 GPU: 4,913.33 EUR
NVIDIA Tesla M40 24GB GPU, Cust Kit: 6,458.95 EUR
NVIDIA M60 GPU, Passive, Requires GRID 2.0 SW for VDI Function, Cust Kit: 5,094.95 EUR
Licenses:
Nvidia GRID vApps Subscription License 3 yr, 1 CCU: 20 EUR
Nvidia GRID vPC Subscription License 3 yr, 1 CCU: 95 EUR
Nvidia GRID vWS Subscription License 3 yr, 1 CCU: 480 EUR
So, if you are ready for at least a 2-year contract to rent a Dell R730 server (not xd, because the solution is much more expensive) - contact sales@ua-hosting.company, we will be happy to help you! Using one of the recommended video cards in the configuration DELL R730 2 x E5-2650 v4 / 128GB / 6 x 480GB SSD / 1Gbps 100TB + GPU and a 2-year contract, it will cost approximately $ 6,816 per year, instead of $ 2,988 per year in the case The Dell R730xd + platform article will require payment of the cost of a video card and license, as this is very specific equipment.
Nevertheless, even this price is much more attractive than the prices that data centers in Ukraine and Russia can offer for such solutions, if they ever want to provide such solutions ... For, oddly enough, in the Netherlands, where cooperation with Dell is debugged directly, the promised the delivery time of the server with the video card was almost 2 months from the time the order was placed by our client (the client agreed to expect, since there are simply no alternatives on the market) , due to the strong specificity of this product, it was simply not available from Dell itself . Nevertheless, the delivery was made on the basis of faster than in a month. Thanks to the data center supply department and the guys from Dell for their promptness. But for me, yet this experience has become unusual, since the standard Dell R730xd platforms are shipped within a few days.
Management and features of the PERC controller
Server management can be accomplished through a convenient iDRAC8 (Integrated Dell Remote Access Controller 8) with a Dell life cycle controller, which reduces the time spent on management tasks, reduces the likelihood of errors, improves security and improves the overall efficiency of your IT environment.
For our subscribers, access to the iDRAC8 is provided via RMI (Remote Management Interface), which is available exclusively from the private data center network for security purposes, access to which we provide through the Open VPN tunnel for free. After logging in, the iDRAC displays an overview of the system, as well as viewing the virtual console via iKVM.
I would like to pay special attention to monitoring. IDRAC8 allows you to get power consumption statistics for the last hour, day or week and limit the maximum power consumption:
The iDRAC temperature monitoring and recording system allows you to collect data from temperature sensors and determine the range in which it is located. Thus, it is considered acceptable to stay processors in the Warning temperature range only for 10% of the time during the year, and critical - for 1%. Time in the critical band also affects the allowable time spent in the warning band. The collection of temperature data begins when the system is turned on after being shipped from the factory and cannot be reset.
Dell's 13th generation PowerEdge servers are equipped with PERC9 controllers, in which the LSI CacheCade technology previously used in PERC8 controllers has been replaced with a more efficient one — SanDisk DAS Cache.
In the course of synthetic tests, DAS Cache gives a significant acceleration of the performance of HDD arrays in cases of hybrid configurations of HDD + SSD. So, in the case of an array of 5 HDDs in RAID6 (the 6th level is selected for maximum capacity) and 5 SSDs RAID10 (4 + 1 hot spare SSDs for maximizing the performance of the array) using DAS Cache, the performance of the array is 5 HDD RAID6 + DAS Cache on the specified SSD array turned out to be close to the performance of the SSD array itself:
So, having a little familiarized with the platform, now we can consider the advantages of this platform when building storages.
What makes the Dell R730xd platform optimal for data warehousing, in particular Red Hat Ceph?
Red Hat Ceph Storage uses industry standard servers for scalability, resiliency and performance. Data protection techniques play a key role in solution costs. Ceph allows the user to select different protection methods for different storage pools.
Replicated storage pools produce complete copies of stored objects and are ideal for quick recovery and data handling. In the case of a replicated storage pool, Ceph defaults to a replication ratio of three when three copies of data reside on three separate Ceph nodes.
Storage pools with protection against data corruption represent a single copy of data with parity, which is useful in case of need for durable data storage and is cost-effective.
The PowerEdge R730xd is an extremely flexible and scalable dual-bay rack server that provides high-performance processing and a wide range of local storage capabilities with workload optimization, including the ability to use hybrid levels. The development includes a wide range of configurations, which is ideal for Ceph.
- R730xd provides the ability to create highly efficient infrastructure solutions for both small and large businesses.
- R730xd can reduce the time of commissioning due to the possibilities of automated deployment, which reduces the number of actions by the user and reduces the number of human errors.
- PowerEdge servers improve the performance of IT infrastructure in data centers using innovative management tools such as iDRAC Quick Sync and iDRAC Direct, which provides the ability to monitor the state of the system comprehensively and perform rapid deployment.
- In PowerEdge servers, energy usage is optimized, which provides better performance per Watt of power consumption, and also allows you to more accurately control energy consumption and cooling.
The Dell PowerEdge R730xd delivers the benefits of maximum performance by:
- Accelerate application efficiency by using the latest technology and dynamic local storage.
- Scaling is easy and fast using front-mounted storage, which supports drives from a wide range - from low-cost SATA HDDs to ultra-fast 2.5 "SSDs, as well as very low latency drives, such as PowerEdge Express Flash NVMe PCIe SSDs.
- R730xd is well adapted for building hybrid-type storages thanks to the use of new caching technologies in the PERC controller, which increases the economic efficiency of the Ceph solution.
An example of infrastructure implementation with a Red Hat Ceph Storage storage, which consists of 5 Dell R730xd servers, you can see below:
The following independent results were obtained during independent tests of various configurations of the Dell R730xd platforms:
Our version of the platform shows a very good result, and in the case of reading, perhaps the best. And if we take into account the fact that we ordered these platforms in large quantities and due to this we can offer a price more than 2 times lower than on the other platform options - it turns out to be the best consolidated for reading and writing on the cost-effective parameter. In the end, nothing prevents to build a cluster of 7-8 platforms, instead of 5. At the same time, to get even greater fault tolerance.
Are 10 Gigabit Switches Required When Building Such Infrastructures? No, not mandatory, one of our clients built the infrastructure, though from 3 servers, using the following local area network connection:
That allowed him to save not a lot of money - $ 350 / month for renting a switch, since 10-Gigabit switches are quite expensive, and we provide only the best - Arista 7050tx-48-r. In decisions on several ports we see no point.
What are the Dell R730xd platforms basically capable of?
Not so long ago, Microsoft conducted an interesting test of the performance of these platforms in the case of building data warehouses with a small number of nodes.
Four Dell R730xd nodes were used, combined into a local 100 Gigabit network using the 32-port Arista DCS-7060CX-32S 100Gb switch, running EOS version 4.15.3FX-7060X.1.
The configuration of the nodes used is as follows:
2x Xeon E5-2660v3 2.6Ghz (10c20t)
256GB DRAM (16x 16GB DDR4 2133 MHz DIMM)
4x Samsung PM1725 3.2TB NVME SSD (PCIe 3.0 x8 AIC)
Dell hba330
4x Intel S3710 800GB SATA SSD
12x Seagate 4TB Enterprise Capacity 3.5 ”SATA HDD
2x Mellanox ConnectX-4 100Gb (Dual Port 100Gb PCIe 3.0 x16)
Mellanox FW v. 12.14.2036
Mellanox ConnectX-4 Driver v. 1.35.14894
Device PSID MT_2150110033
Single port connected / adapter
With VMFleet, 20 virtual machines were raised on each node, that is, only 80 virtual machines. Each virtual machine was configured with 1vCPU. Then VMFleet was used to run the DISKSPD utility to test storage performance on each of the 80 nodes, the utility is available here for free. Test parameters are as follows: 1 stream, 512KiB sequential reads for 4 I / O operations.
As you can see in the image above, it turned out to achieve an incredible result of 60 gigabytes per second in total, which is approximately equal to 5 English versions of Wikipedia in compressed form (11.5GiB), which are downloaded every second. And the speed from each virtual machine was about one CD per second - 750 MB.
This test perfectly demonstrates how effective a solution can be when the three components — computing power, storage, and network — are balanced, which minimizes the potential bottlenecks that can occur in an unbalanced system.
What about platforms from other manufacturers?
Comparison of Dell R730xd and HP ProLiant DL380
For comparison, the performance was chosen iron weaker than is offered for rent by us - servers with an E5-2620v3 processor, and not E5-2650v4, which are much more productive. The main testing methodology used for this comparison was to measure the amount of IOPS. Several different workload simulations were used, namely, all Read and 30% Read / 70% Write (an analogue of an OLTP system, a transactional system, when processing goes in small transactions, but with a large flow, and customers need to ensure a minimum response time).
Tests were conducted several times to eliminate the negative impact of hidden factors on system performance. First, the base tests were carried out on the Dell R730xd and HP ProLiant DL380 using only HDD drives in the storage (5x1TB HDD RAID5), which served as a benchmark for performance. Then, the same test suite was launched with 5x1TB HDD RAID-5 storage using DAS cache installed on SSDs with SAS interface (2xSAS 480GB Samsung SSD RAID1) on the Dell server, and on the HP server using HPE SmartCache with similar set of drives for storage and caching. Finally, the latest test suite was launched on a Dell server using DAS Cache on NVMe PCIe SSDs (2x400GB Samsung NVMe in Software RAID) to display the maximum contribution of caching to application performance gains in cases of intensive read and write. A similar test for the HP server was not performed, because Smart Cache does not support NVMe disks for caching.
System configuration:
| Dell PowerEdge R730xd (13 Generation) | HP ProLiant DL380 (9 Generation) | |
|---|---|---|
| Server | CPU: x86-64 - Intel Xeon CPU E5-2620 v3 @ 2.40GHz; Memory: 32GB DDR4. | CPU: x86-64 - Intel Xeon CPU E5-2620 v3 @ 2.40GHz; Memory: 32GB DDR4. |
| Operating system / software | Windows 2012 R2 SP1; SanDisk DAS Cache v1.4. | Windows 2012 R2 SP1; HPE SmartCache. |
| Storeage / Cash | 5x1TB HDD RAID5; 2xSAS 480GB Samsung SSD RAID1; 2x400GB Samsung NVMe Software RAID. | 5x1TB HDD RAID5; 2xSAS 480GB Samsung SSD RAID1. |
| Test | Simulate OLTP workload for read-write; 300GB database size; 4 simultaneous IOMETER workers with a queue depth of 32. | Simulate OLTP workload for read-write; 300GB database size; 4 simultaneous IOMETER workers with a queue depth of 32. |
Several random read / write tests were conducted with different block sizes - 4 and 8KB. 100% , 70% 30% . , , 900 (15 ), , .
IOPS ( / ):
| / | 4KB RR 100% Read | 4KB RR 70% Read 30% Write | 8KB RR 100% Read | 8KB RR 70% Read 30% Write |
|---|---|---|---|---|
| Dell R730xD ( ) | 1650 | 974 | 1540 | 1040 |
| HP ProLiant DL380 ( ) | 1370 | 628 | 1322 | 630 |
| Dell R730xD DAS Cache | 138884 | 66483 | 98368 | 56641 |
| HP ProLiant DL380 SmartCache | 41273 | 33534 | 35984 | 39396 |
| Dell R730xD DAS Cache NVMe PCIe SSD | 264750 | 158157 | 257150 | 104490 |
% :
| / | 4KB RR 100% Read | 4KB RR 70% Read 30% Write | 8KB RR 100% Read | 8KB RR 70% Read 30% Write |
|---|---|---|---|---|
| Dell R730xD ( ) | 0.4% | 0,28% | 0,42% | 0,3% |
| HP ProLiant DL380 ( ) | 0,8% | 0.5% | 0,8% | 0.5% |
| Dell R730xD DAS Cache | 13% | 8,8% | 11,34% | 7,83% |
| HP ProLiant DL380 SmartCache | 6% | 6% | 5% | 5% |
| Dell R730xD DAS Cache NVMe PCIe SSD | sixteen% | 10,1% | sixteen% | 5,78% |
Dell R730xd SanDisk DAS cache HPE SmartCache, NVMe PCIe SSD . OLTP R730xd IOPS 3- SanDisk DAS Cache SAS SSD 6 SanDisk DAS Cache + NVMe PCIe SSDs HPE's SmartCache. SanDisk DAS Cache, SanDisk DAS Cache + NVMe PCIe SSDs IOPS .
, HP 6% IOPS 41 , 1 IOPS 0,15% , Dell R730xd 16% 264 , — 0,06% 1000 IOPS.
, , Dell ( 2,5 ), % , , — , .
Dell R730xd HP ProLiant DL380 ( ), , cost-effective.
Geekbanch 3 Dell R730xd c 5-2640 v4
, , — , , , .
. , , , , . , , 5-2650 v4, , 70% ( AES), 5-2650 v3.
«» low-ost, ? ? ? — . , . - , latency , , latency !
, 10 , 9 , , . 5-2640 v4, 10, 12 , .
https://browser.primatelabs.com/v4/cpu/768278 — Geekbanch 3 Dell R730xd c 5-2640 v4
, , , , SanDisk DAS cache, , , ! - .
. AES, 1- 1000 , , , 100 .
, , , , .
, , , .
Dell R730xd, Bare Metal, simply better!
VMware Bare Metal ( ) Dell R730xd . , .
? - . , , - ( ) — . , , , , , … , , , , , , , , , 5 - … , , « » - - .
, . , :
, 15-30 .
100%, , 100%. 99,9% , . 0,1% — 44 , — . 99,99% , 4 . 2 , , . , 40 , . . 5 - , . blackout - — - , - , - . , , , , , , 3 , , 3 . 30 3 ? , .
, ? , . , : , - . , 40 , , — . 40 , 40 — , . , .
, , , .
. bare metal Dell , , , , , . , cost-effective .
. — , . , , , 3 , , . Dell R730xd , . , VMware.
Dell R730xd
Many of our customers who use these servers for databases have really liked this platform. And not only because, unlike the Dell R730, the xd version can be provided with support for NVMe PCIe SSDs, which provide minimal latency, which is extremely important for such solutions. But also because the controller is incredibly effective at work even with the usual 2.5 "SSD, which we offer to customers.
Of course, there is a drawback to the Dell R730xd - the controller is only 1, while in the Dell R730 there are 2 hardware controllers for 26 drives. But this is not a significant problem, since we chose a platform with 12 compartments for storage and therefore you will not get a bottleneck in the form of insufficient controller performance when working with this platform and a large number of SSDs. And the advantage in the form of the possibility of installing the NVMe PCIe SSD fully covers this disadvantage.
As mentioned above, not everyone needs complex database clusters, the performance that this platform can provide is much more important. One of our clients did just that, abandoned the excesses of virtualization and building complex clusters, renting a VPS with dedicated drives for database backups from the main Dell R730xd server: VPS (KVM) - E5-2650 v4 (24 Cores) / 40GB DDR4 / 4x240GB RAID10 SSD 1Gbps 40TB - $ 99. Of course, in case of a problem with the main node, there will be a slight decrease in performance and the need for manual switching will arise, however, this is more than a cost-effective solution, given the limited budget.
We do not call everyone for such savings, in some cases it is even very justified. However, by increasing the budget by only $ 230 per month, you can allow the E5-2650 v4 node that is already identical and connect them together to a 20-gigabit local area network, which in principle allows you to build a small cluster with acceptable latency even without using PCIe NVMe SSDs.
The solution itself will look like this (with a basic set of drives):
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $ 289 / month
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $ 289 / month
Increasing their number to increase performance does not always make sense. So, during tests with databases, we found that a solution with 8 SSDs is more efficient than a solution with 4 SSDs by only 35%.
Regarding the recommended RAID levels, in some cases RAID5 can be very optimal. Of course, RAID10 provides better performance, however, what if the drives are so fast that using them in RAID10 can lead to a controller becoming a bottleneck? In such cases, building a HW RAID RAID5 array is much more profitable using hot-spare drives than RAID10 (if the goal is not to lose half the SSD quota).
Do I need the controller cache to be on or better off? The RAID controller has a very limited amount of memory. What is 1GB in case of working with SSD? In cases with fast SSDs, there is no sense in the included cache for reading, since filling memory with read operations that will not become faster at all, since SSDs are fast enough anyway, we don’t leave enough cache for writing, and, as we know, in inexpensive SSDs - the bottleneck is the write operation, due to a drop in performance with insufficient over-provisioning and 100% utilization of the free quota (perhaps a drop tenfold). Therefore, we recommend leaving only the write cache enabled. Thereby ensuring maximum effectiveness of this solution.
Is NTFS block size so important? For most storage, 64KB is listed as the optimal block size for maximum performance. But very often we are faced with cases where the client is already using the server for the databases, applying the initial configuration with the standard NTFS cluster size of 4K. To fix this, you need to reformat the drives, but is it really a big difference in the general case? Do I need to expose myself to the possible downtime and inconvenience associated with migration?
The controller in the Dell R730xd server provides so good performance even with 4 SSD drives and a write cache enabled (a very important point, the read cache must be disabled because it does not provide any benefits, but only reduces the efficiency of write operations due to the lack of space for full cache, as indicated above), even with a standard block size of 4K, we get very good performance indicators, which are extremely good.
Nevertheless, we always go to a meeting and are ready to offer a similar platform for a period of time for work by subscribers, if something was not taken into account by them.
RAMdrive still turns out to be faster than SSD drives. If your requests can be processed in TempDB, for example, sorting or merging, then using RAMdisk (a program that turns part of your RAM into a drive) can be quite justified. So, comparing the speed of 8 SSD drives in RAID10 with RAMdrive, we found that it is about 4 times faster with 4K write operations with queue depth of 32, which simulates the operation of TempDB files. Even combining 8 SSDs in RAID0 failed to beat the results of RAMdrive.
RAID5 in the case of 8 SSDs may be more efficient than RAID10. 4K read operations with a queue depth of 32% faster by about 40%, and this is not surprising, given the features of RAID5. RAID5 wins in performance in other tests, so with sequential writing it is 20%, is it inferior only with 4K write operations with a large queue depth (in our case 32) and turns out to be less effective by 30%. So the use of RAID5 can in some cases be very cost-effective.
More storage does not always mean faster. When we deal with random small operations, adding drives to an array after a certain limit does not improve performance. It would seem why? It turns out everything depends on the capabilities of the controller, which is forced to record data on a whole bunch of drives. That is why we purchased platforms with a maximum of 12 drives in order to reduce the likelihood of a bottleneck. However, you need to look at the intensity of operations within each project, sometimes 4 larger drives, say 960GB, can be a better and more cost-effective solution than a bunch of 480GB drives.
Using Dell R730xd Servers for Aerospike NoSQL Data Servers
Aerospike is an open source NoSQL database optimized for flash memory, providing high speed and scalability. Aerospike is designed to provide very low latency for read requests and a large write load, which satisfies the current business requirements of those who need to handle a large number of I / O operations. Aerospike directly manages local storage on the nodes of its cluster, regardless of the source file systems, be it DRAM, flash or traditional hard drives. This allows you to store critical data on the fastest media, the fastest processing of which is most critical, while the distributed nature of the Aerospike database provides a high degree of reliability in the event of a cluster node failure.
The 13th generation Dell servers, namely the PowerEdge R730xd, with the latest E5-2650 v4 processors and fast DDR4 SDRAM, provide incredible computing power. And the ability to use NVMe SSDs, such as the Samsung SM1715, can help get a lot of performance in nodes with high loads with minimal latency. That allows you to meet modern SLA requirements for the processing of a single transaction with a reliable level of fraud protection even in very large databases, making the decision acceptable even for the financial sector and bank transfers, where transactions are sometimes carried out within a few hours or even days.
For the banking sector, we can offer a solution based on these platforms with a local network of 40 and 100 Gbit / s between the cluster nodes and the use of appropriate switches.
Using Dell R730xd Servers in DMZ Networks
We are pleased to offer our subscribers the highest security standards. The data centers in which we offer these servers have many certifications: ISO 27001 , PCI DSS , SOC 1 , HIPAA and NEN 7510 .
But in order to meet the requirements of the same Google, some clients need to build so-called demilitarized networks (DMZ networks) - networks with enhanced security, which implies a hardware firewall that restricts access to equipment and allows you to hide the necessary components behind the firewall and increase safety more.
And everything is ok, when such a solution is ordered immediately, but it is very difficult to implement a transition almost without downtime in the process that recently happened to one of our clients, with a project to process applications for visas for citizens from different countries of the world, where it is not available even during hours can result in losses of tens of thousands of dollars. But we found a solution, went to the meeting, put the additional equipment in the closet, moving less important, and included the firewall according to the scheme, as the client wished, so that he could migrate without downtime.
It was / It became
Perhaps this scheme will also be useful for someone to carry out such a migration, as engineers in data centers do not always want to see obvious things, but insist on downtime at 2 o'clock to switch to the firewall, but I beg you - order a hardware firewall all well in advance if you need it. We are flexible, and always go to a meeting, but in some situations our opportunities may be limited.
In future publications, we will try to conduct a series of additional tests and share additional experience related to the use of these platforms, which will further enhance the understanding of the compilation of the most effective solution. After all, it is surprising to see the fact that RAID0 is from two SSD drives, in cases of high load databases, it may turn out to be slower than a single write drive, that is why we receive requests for NVMe PCIe SSD from time to time. It is likely that we will show the most effective scenarios for implementing solutions, share additional experience when building multiple RAID arrays within a single RAID controller turns out to be effective, show in which cases two independent RAID10 arrays from 4 drives will be better than one RAID10 array out of 8. When will it be efficient to use RAID1 for TempDB, and RAID5 for everything else. And how can you do without NVMe PCIe SSD, when there is a limited budget.
Dell R730xd: bonus for Habrahabr readers
We want to provide you with the best and have done everything for this by offering unique prices for the Dell R730xd:
ua-hosting.company/serversnl - in the Netherlands
ua-hosting.company/serversus - and in the USA
Configurations can be very diverse, for example, the following are available for instant activation in the Netherlands :
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12x240 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 4x4TB 4x480 SSD 1Gbps 100 TV - * $ 249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12x4TB SATA 1Gbps 100 TV - * $ 249
And honest dedicated 1Gbps Unmetered (without traffic) is available to them for + $ 120 / month. Plus the ability to purchase a firewall, network cards, a switch and build a multi-gigabit local area network. But the condition of the price is valid for a year contract.
We want to improve the condition and offer a bonus for short-term payment and rent without a contract, making the offer more affordable. For anyone who orders and pays for any of the proposed options in the Netherlands for a period of 1 month , we will provide the exact same server for 1 month completely free of charge , or we will recalculate the ordered configuration and calculate the ordered server at a price as with an annual contract ($ 249 / month instead of $ 369 / month), returning the difference to the balance. To do this, simply leave the order number in the comments. For those who have not yet reached dedicated solutions, we suggest renting any VPS (KVM) with dedicated drives in the Netherlands or the USA on these platforms, for example, VPS (KVM) - E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps 10TB - $ 29 , and get a bonus of 1-4 months of use when paying for 1, 3, 6, 12 months, respectively, with the number of the order in the comments. Now build infrastructure has become easier!
Source: https://habr.com/ru/post/329618/
All Articles