The history of assembling a “village supercomputer” from spare parts from eBay, Aliexpress and a computer store. Part 1
Good day, dear Habrovchane!
I want to tell you a long and, as I hope, fascinating, and maybe even useful, history of the assembly of the “village supercomputer” from the board of the Dell server node, the Nvidia Tesla K20 GPU, and the fact that it was bought here and there at all kinds of online stores or in the computer stores of your city.
The story began with the fact that my friend is a programmer, who is also an astronomer at the same time - he began studying neural networks. Their "full-time specialist" resigned and the topic was hung on the "closest specialist" himself. I myself am not a programmer, just “a radiomechanic for the repair of computer technology (c) my diploma”, because assembling all sorts of interesting computer hardware is an interesting and enjoyable task for me. I work unfortunately in another area.
In order to formulate the task more clearly, I created a topic on the “Iron Ghosts of the Past” forum where it was discussed for a long time. At first there was a rather naive idea of “assemble a 4-way SLI on a GTX 580 3Gb”, which gradually transformed into an understanding - you need to assemble a server! Prices for server motherboards shamelessly bite until I came across Youtube on an interesting video about launching a Chinese server board on 2 non-standard format processors.
')
Here is a video:
Especially in this video I was pleased with the budget price of the system.
However, consultation with more knowledgeable comrades who dealt with Chinese servers convinced me - “We don’t need Chinese happiness!” According to their comments, Chinese servers were simply terribly unreliable. And I began to look for Avito options with Dell server boards. I have two laptops from this company and from them only positive impressions. Very reliable technique.
On Avito, the board of the Dell PowerEdge C6220 server node was found in the process of communicating with the seller of which - he suggested to me an excellent site where there was a publication about how one craftsman ran such a board, here is the link . And there was a link to the American forum, where such boards collected powerful workstations. This topic is here .
I read the whole topic from the beginning to the end, I decided on the goals, objectives and ways to achieve them. The task was formulated as follows: “Build a dual-processor server on a Dell PowerEdge C8220 node with a Tesla K10 or K20 GPU”. The choice of specialized GPUs fell after discussion with the person for whom the system was actually going - having “maps” capable of conducting long-term computations with double precision and monitoring ESS memory errors, he could use them for his scientific work, and not just for training neural networks. What he actually very happy.
For discussion and impressions for the history of the assembly process at the Iron Ghosts of the Past forum, I created a corresponding topic, where I actually wrote about the process and shared photos. Those interested can be found .
The task was set and I started searching for components. At the time when it all started, I didn’t have a registration on eBay and, at the beginning, the necessary parts were bought by my friends, with whom I paid the costs of purchasing and shipping. Later, I myself registered there and began to buy directly, although sometimes I have to ask for help from those who have accounts on Shopotame and similar services. Not all necessary parts are sent directly from the USA to Russia.
The first was a Dell PowerEdge C8220 0083N0 motherboard. According to Dell's documentation, it related to the motherboard version 1.2 having 3 PCI-E 16x slots. The two usual power buttons and the third on the other side of the board are non-standard, under the so-called GPGPU riser, which was included in the so-called Edge Slot.
Photo board, the very 0083N0, photo from eBay.
And this is already my photo, a ruler is attached to the board to realize the scale.
By then, the GPGPU riser arrived at the Edge Slot for me.
Here is a photo where it is connected to the sample at its regular place.
At the same time, a power adapter was purchased on eBay, from ATX to this power connector C6100. They are sold on eBay two types, 12 and 18 pin. We need the latter, and another DC-DC boost to turn + 5VSB from ATX PSU into Dell server + 12VSB. And, of course, the mama response pad is in the slot to install the jumper needed to start the board and output the PS_ON signal from it. By the way, it has a non-standard contact pitch of 2.0 mm. Of course, desperate guys can weave a screwdriver or a nail right into the board connector, but I preferred to do everything culturally.
In addition, the cheapest Xeon E5-2604 V1 was purchased for a trial launch of the motherboard from Aliexpress and a pair of DDR3 ECC REG memory bars that were sold as compatible with the Dell PowerEdge C8220 were purchased from eBay. I used Alpine 20 Plus C0 coolers for LGA 2011, which had to be finalized - with a grinder, the grinder of their edges rested in memory slots, spring washers were removed from the fastening screws, and two screws were screwed on the threads to prevent the screws from being screwed too deep and do not break through the board. Server sockets LGA 2011 are arranged slightly differently than usual and radiator screws have a short thread there. By the way, the coolers worked well, despite the fact that they were purely aluminum.
And so, that moment came when the processors arrived, I captured them on the memory photo.
And here are the very same Alpine aluminum coolers.
Build and run system.
My old faithful Chiftek 550 W power supply unit was connected to the system, a USB hub for 4 devices that included a keyboard, a mouse and a USB flash drive with Ubuntu, a card reader into which I plugged a Chinese USB sound adapter was connected to the USB card reader connector, I also connected a VGA monitor and a patchcord to the IPMI 100 Mbps port, which is called Delicated-NIC. Next to it are two 10Gbe-ports that work on the usual twisted pair of "copper" and fully support the usual 100/1000 network.
In this form, the system was launched and it turned out that the board checks the memory for a very long time at startup. And in the BIOS splash screen, she called herself the Dell DCS 6220.
With this, I will finish the first part of my story so as not to bore grateful readers.
Link to part 2: habr.com/en/post/454448
I want to tell you a long and, as I hope, fascinating, and maybe even useful, history of the assembly of the “village supercomputer” from the board of the Dell server node, the Nvidia Tesla K20 GPU, and the fact that it was bought here and there at all kinds of online stores or in the computer stores of your city.
The story began with the fact that my friend is a programmer, who is also an astronomer at the same time - he began studying neural networks. Their "full-time specialist" resigned and the topic was hung on the "closest specialist" himself. I myself am not a programmer, just “a radiomechanic for the repair of computer technology (c) my diploma”, because assembling all sorts of interesting computer hardware is an interesting and enjoyable task for me. I work unfortunately in another area.
In order to formulate the task more clearly, I created a topic on the “Iron Ghosts of the Past” forum where it was discussed for a long time. At first there was a rather naive idea of “assemble a 4-way SLI on a GTX 580 3Gb”, which gradually transformed into an understanding - you need to assemble a server! Prices for server motherboards shamelessly bite until I came across Youtube on an interesting video about launching a Chinese server board on 2 non-standard format processors.
')
Here is a video:
Especially in this video I was pleased with the budget price of the system.
However, consultation with more knowledgeable comrades who dealt with Chinese servers convinced me - “We don’t need Chinese happiness!” According to their comments, Chinese servers were simply terribly unreliable. And I began to look for Avito options with Dell server boards. I have two laptops from this company and from them only positive impressions. Very reliable technique.
On Avito, the board of the Dell PowerEdge C6220 server node was found in the process of communicating with the seller of which - he suggested to me an excellent site where there was a publication about how one craftsman ran such a board, here is the link . And there was a link to the American forum, where such boards collected powerful workstations. This topic is here .
I read the whole topic from the beginning to the end, I decided on the goals, objectives and ways to achieve them. The task was formulated as follows: “Build a dual-processor server on a Dell PowerEdge C8220 node with a Tesla K10 or K20 GPU”. The choice of specialized GPUs fell after discussion with the person for whom the system was actually going - having “maps” capable of conducting long-term computations with double precision and monitoring ESS memory errors, he could use them for his scientific work, and not just for training neural networks. What he actually very happy.
For discussion and impressions for the history of the assembly process at the Iron Ghosts of the Past forum, I created a corresponding topic, where I actually wrote about the process and shared photos. Those interested can be found .
The task was set and I started searching for components. At the time when it all started, I didn’t have a registration on eBay and, at the beginning, the necessary parts were bought by my friends, with whom I paid the costs of purchasing and shipping. Later, I myself registered there and began to buy directly, although sometimes I have to ask for help from those who have accounts on Shopotame and similar services. Not all necessary parts are sent directly from the USA to Russia.
The first was a Dell PowerEdge C8220 0083N0 motherboard. According to Dell's documentation, it related to the motherboard version 1.2 having 3 PCI-E 16x slots. The two usual power buttons and the third on the other side of the board are non-standard, under the so-called GPGPU riser, which was included in the so-called Edge Slot.
Photo board, the very 0083N0, photo from eBay.
And this is already my photo, a ruler is attached to the board to realize the scale.
By then, the GPGPU riser arrived at the Edge Slot for me.
Here is a photo where it is connected to the sample at its regular place.
At the same time, a power adapter was purchased on eBay, from ATX to this power connector C6100. They are sold on eBay two types, 12 and 18 pin. We need the latter, and another DC-DC boost to turn + 5VSB from ATX PSU into Dell server + 12VSB. And, of course, the mama response pad is in the slot to install the jumper needed to start the board and output the PS_ON signal from it. By the way, it has a non-standard contact pitch of 2.0 mm. Of course, desperate guys can weave a screwdriver or a nail right into the board connector, but I preferred to do everything culturally.
In addition, the cheapest Xeon E5-2604 V1 was purchased for a trial launch of the motherboard from Aliexpress and a pair of DDR3 ECC REG memory bars that were sold as compatible with the Dell PowerEdge C8220 were purchased from eBay. I used Alpine 20 Plus C0 coolers for LGA 2011, which had to be finalized - with a grinder, the grinder of their edges rested in memory slots, spring washers were removed from the fastening screws, and two screws were screwed on the threads to prevent the screws from being screwed too deep and do not break through the board. Server sockets LGA 2011 are arranged slightly differently than usual and radiator screws have a short thread there. By the way, the coolers worked well, despite the fact that they were purely aluminum.
And so, that moment came when the processors arrived, I captured them on the memory photo.
And here are the very same Alpine aluminum coolers.
Build and run system.
My old faithful Chiftek 550 W power supply unit was connected to the system, a USB hub for 4 devices that included a keyboard, a mouse and a USB flash drive with Ubuntu, a card reader into which I plugged a Chinese USB sound adapter was connected to the USB card reader connector, I also connected a VGA monitor and a patchcord to the IPMI 100 Mbps port, which is called Delicated-NIC. Next to it are two 10Gbe-ports that work on the usual twisted pair of "copper" and fully support the usual 100/1000 network.
In this form, the system was launched and it turned out that the board checks the memory for a very long time at startup. And in the BIOS splash screen, she called herself the Dell DCS 6220.
With this, I will finish the first part of my story so as not to bore grateful readers.
Link to part 2: habr.com/en/post/454448
Source: https://habr.com/ru/post/454282/
All Articles