Experience of transition to the DNS-server NSD
Back in 2007, when Ukrnames was still small and took the first steps, most of the work was done by programmers. They developed the domain management system, tested it, and administered the servers where all this stuff worked.
And where there are domains, there is DNS. And what is the easiest way to make a DNS server? Put the omnipresent BIND. Yes, BIND is not easy, but with the DLZ patch so that it can be friends with MySQL.
The years went by, the system worked, the server power grew. And as usual, while working - do not touch. So it would all continue if the load did not start to rest against the next 8th nuclear server.
')
At that moment, I had to make a volitional decision and build a new system of DNS servers, instead of upgrading the hardware for the insatiable BIND + DLZ again.
And here the fairy tale ended, and the harsh everyday life of the administrator began.
It is necessary to maintain a service that Ukrnames for domains provides for free, namely “Managed DNS”.
Currently this service is used for more than one hundred thousand domains. Domains are diverse, with different attendance, respectively, the load does not depend on the number of domains.
The service is served by 3 DNS servers, with Xeon X3430 configuration, 8Gb DDR3, 2x500Gb SATA RAID1. Servers use BIND + DLZ patch in conjunction with MySQL. The database works as a slave of the currently active domain zone storage servers.
From the existing variety, PowerDNS with MySQL and NSD (work on files) were chosen.
It was decided to make a competition for the old BIND and these two impostors.
At first glance, the most logical choice is to take a DNS server that out of the box knows how to work with the MySQL database and bind it to the existing database.
In practice, it turned out that with the existing database and with the one that requires PowerDNS, there were several rather unpleasant differences.
As a result, for implementation, it was necessary to either change the database structure, and this will pull changes in the code of the entire system. Either build crutches, namely procedures or view.
For the competition was chosen the second option, because The main system could not be touched.
Judging by Wikipedia , NSD successfully works on 3 root servers.
A feature of NSD is that it compiles all zones into its internal database format, and then loads it entirely into RAM.
On the developer's site there is a tool for calculating the consumption of RAM . In our case we got about 367 Mb, which was more than an acceptable result.
A utility was written that sent requests to the DNS server in 50 threads. The domains were chosen randomly from the existing database, the type of records also changed randomly.
For PowerDNS, a view was written that gave the data in the correct format. During testing, it was rewritten 3 times for optimization.
For NSD, a utility converter was written on my favorite Google Go that sampled the entire database and compiled the database into the internal NSD format.
In both cases, logical filters were used to convert all inconsistencies with the RFC into a digestible form.
Also, the base was cleaned of perennial debris. Without idiomatic expressions has not done. I never would have thought that SUCH can be recorded in DNS. There were both URLs in CNAME records, and emails with passwords in MX. Apparently, programmers initially believed in users too, and did not make normal filters for recordings.
In turn, a test was conducted for each participant. Test duration 10 minutes. Once a minute, the average response time for the last minute and the current processor load in percent are stored. Testing was performed on a machine with the same configuration as the combat DNS servers.
PowerDNS was configured to generate SOA on its own to reduce the number of queries to MySQL, the cache of records was also turned on for 1 minute.
NSD worked in 4 threads, in a standard configuration.
BIND + DLZ showed itself as usual, a huge response time, space load.
PowerDNS showed good performance, as well as BIND, giving approximately 50 to 50 loads for the main daemon and MySQL. And he would be the leader if there was no NSD in the test ...
Response time:
CPU load:
The NSD test was repeated 3 times, because there was a persistent feeling that something was wrong, and this could not be.
Actually the result suggests itself.
MySQL logic is a weak point. It is better to process such data already on the recipient and make as few requests as possible.
For DNS is quite normal if the data is updated no more than half an hour. The converter utility runs in 7 seconds and allows for a minute update.
NSD now works on our servers and we will not think about iron upgrade soon, except to replace the disks with more recent ones.
And where there are domains, there is DNS. And what is the easiest way to make a DNS server? Put the omnipresent BIND. Yes, BIND is not easy, but with the DLZ patch so that it can be friends with MySQL.
The years went by, the system worked, the server power grew. And as usual, while working - do not touch. So it would all continue if the load did not start to rest against the next 8th nuclear server.
')
At that moment, I had to make a volitional decision and build a new system of DNS servers, instead of upgrading the hardware for the insatiable BIND + DLZ again.
And here the fairy tale ended, and the harsh everyday life of the administrator began.
What we have?
It is necessary to maintain a service that Ukrnames for domains provides for free, namely “Managed DNS”.
Currently this service is used for more than one hundred thousand domains. Domains are diverse, with different attendance, respectively, the load does not depend on the number of domains.
The service is served by 3 DNS servers, with Xeon X3430 configuration, 8Gb DDR3, 2x500Gb SATA RAID1. Servers use BIND + DLZ patch in conjunction with MySQL. The database works as a slave of the currently active domain zone storage servers.
We are looking for problems on our ... head
- High load, 100% server capacity is used. Half falls on MySQL, the second half falls on BIND.
- DLZ patch is already obsolete and does not develop. Infinitely redoing a patch for new versions of BIND is not the best approach.
- When using other solutions, the problem of the old database arises, which was initially filled without any special checks. Lots of junk, lots of entries that don't comply with RFC. Their sane will not accept any sane DNS.
Uncover tambourine
From the existing variety, PowerDNS with MySQL and NSD (work on files) were chosen.
It was decided to make a competition for the old BIND and these two impostors.
Watch PowerDNS
At first glance, the most logical choice is to take a DNS server that out of the box knows how to work with the MySQL database and bind it to the existing database.
In practice, it turned out that with the existing database and with the one that requires PowerDNS, there were several rather unpleasant differences.
As a result, for implementation, it was necessary to either change the database structure, and this will pull changes in the code of the entire system. Either build crutches, namely procedures or view.
For the competition was chosen the second option, because The main system could not be touched.
Feel the NSD
Judging by Wikipedia , NSD successfully works on 3 root servers.
A feature of NSD is that it compiles all zones into its internal database format, and then loads it entirely into RAM.
On the developer's site there is a tool for calculating the consumption of RAM . In our case we got about 367 Mb, which was more than an acceptable result.
Charge the tambourine
A utility was written that sent requests to the DNS server in 50 threads. The domains were chosen randomly from the existing database, the type of records also changed randomly.
For PowerDNS, a view was written that gave the data in the correct format. During testing, it was rewritten 3 times for optimization.
For NSD, a utility converter was written on my favorite Google Go that sampled the entire database and compiled the database into the internal NSD format.
In both cases, logical filters were used to convert all inconsistencies with the RFC into a digestible form.
Also, the base was cleaned of perennial debris. Without idiomatic expressions has not done. I never would have thought that SUCH can be recorded in DNS. There were both URLs in CNAME records, and emails with passwords in MX. Apparently, programmers initially believed in users too, and did not make normal filters for recordings.
Dancing
In turn, a test was conducted for each participant. Test duration 10 minutes. Once a minute, the average response time for the last minute and the current processor load in percent are stored. Testing was performed on a machine with the same configuration as the combat DNS servers.
PowerDNS was configured to generate SOA on its own to reduce the number of queries to MySQL, the cache of records was also turned on for 1 minute.
NSD worked in 4 threads, in a standard configuration.
results
BIND + DLZ showed itself as usual, a huge response time, space load.
PowerDNS showed good performance, as well as BIND, giving approximately 50 to 50 loads for the main daemon and MySQL. And he would be the leader if there was no NSD in the test ...
Response time:
CPU load:
The NSD test was repeated 3 times, because there was a persistent feeling that something was wrong, and this could not be.
Total
Actually the result suggests itself.
MySQL logic is a weak point. It is better to process such data already on the recipient and make as few requests as possible.
For DNS is quite normal if the data is updated no more than half an hour. The converter utility runs in 7 seconds and allows for a minute update.
NSD now works on our servers and we will not think about iron upgrade soon, except to replace the disks with more recent ones.
Source: https://habr.com/ru/post/129711/
All Articles