Detailed TTL Tuning Guide for DNS Records

DNS is a fundamental technology product. Processing virtually all top-level network requests and Internet searches, sending Internet traffic and e-mail, as well as many other operations, are made possible by setting certain matches when searching for DNS (converting names such as some.domain.org to IP addresses or names of other domains).

We decided to write about the time to live (Time To Live, TTL) data, since most system administrators do not have to work with DNS configurations every day and much of the information about this parameter is half-forgotten tales transmitted by system administrators from generation to generation.

After asking the corresponding question on Twitter, we found out that some system administrators cannot even decipher the abbreviation TTL (although such, fortunately, are in the minority).
')
Guess the riddle! What does the abbreviation TTL mean when it comes to DNS?

18:43 Oct 26 2016
4% Border of terminal transfer
85% lifetime
8% Telemetry transport line
3% Trusted Technology List
26 votes • Final result
Retweet 1 Like mark 1

To clarify, we will cover the following topics.

1. Overview of the DNS system and TTL setting
2. Troubleshooting TTL problems in DNS records
3. Recommendations for managing changes in DNS records
4. DNS tools
5. Further actions

1. GENERAL INFORMATION ON DNS SYSTEM

What is a DNS record?

The Domain Name Server (Domain Name Server) DNS entry contains two important parameters:
addressing (matching) requests for a particular entry;
time of entry for caching - this is the meaning that the ominous name “lifetime” (TTL) has received.

Why are DNS records cached?

Many organizations set up DNS records once and after that they have not changed them for years. Because DNS records are often requested, but rarely updated, caching them can significantly improve network performance, increasing the complexity of the assessment and troubleshooting DNS problems.

What is TTL?

Time to Live (TTL) is the duration of the caching of the record * by each link * of the DNS matching chain. This value is measured in seconds (the importance of this will be explained below).

Unfortunately, the generally accepted term “lifetime” cannot be called exhaustive. Perhaps it would be more logical to call the name “search time” or “the duration of the storage of the DNS record in the cache”

What is the standard TTL value for DNS records?

The TTL value is always expressed in seconds. Most DNS configuration services provide a complete list of values ​​for records.
300 seconds = 5 minutes = “Very short”
3600 seconds = 1 hour = "Short"
86,400 seconds = 24 hours = "Long"
604,800 seconds = 7 days = “Absolute Maximum”

How is a DNS lookup performed?

When you enter a URL in a browser, a whole series of searches are created.
At each stage of this process (often there are more stages than listed) the following questions are asked.

1. Is the entry cached?
2. If so, is it a TTL value?

If the answer “No” is received to any of these questions, the request moves to the next link in the chain.

Why is DNS based on network connections, not devices?

Resolving DNS problems is not an easy task, not only because of a number of difficulties associated with the use of TTL and caching systems, but also because many modern devices are connected through various networks and chains of DNS servers.

Consider the situation on the example of an ordinary laptop. I usually work on a laptop at home. Despite the fact that I have not taken it anywhere for several weeks, during this time the following connections were made on the device:

• to the main home network Wi-Fi / cable network;
• to a mobile phone when the cable network is unavailable;
• Both options are higher, but connected via VPN.

Each time the network changes, a new DNS chain is activated. If this happens at the time of the change, the servers and caching nodes in the DNS chain may provide incorrect data.

This often happens in corporate networks where the Active Directory domain name matches the address of the company's website. The external DNS server (Internet provider level) stores the DNS record that directs the address www.example.com to the correct IP address / CNAME of the web server, but the internal DNS server used by Active Directory does not duplicate the records.

A panic will immediately begin: “The web server is not working!”, “This is the end of the world!”, “Where are my pants?”. But, starting to fix the problem, you will find that it was caused by an unclosed connection via VPN.

2. TROUBLESHOOTING WITH TTL IN DNS RECORDS

How long does it take to update a DNS record?

To calculate the maximum (worst case) time interval required to update the value of the DNS record in the links for all clients, multiply the number of links in the chain (excluding the authoritative server) by the TTL value.

For example, if the TTL value is 3600 seconds (1 hour), and the DNS chain consists of 5 links, the full propagation of changes should take no more than 18,000 seconds (5 hours).
But if everything was so simple.

What are the costs of DNS lookup?

When it comes to the “costs” of a DNS search, it usually means not monetary, but time costs. Depending on the number of Internet gremlins in the global network, it usually takes 100–200 milliseconds to complete a DNS query.

This is a very short time, but imagine a web page. The correspondence between the name and the IP address in the DNS system must be configured for all images, CSS files and JavaScript asset files that are accessible via links on the page. Without caching, the boot time will increase significantly.

Simplified DNS Lookup Costing Scheme

I called this scheme simplified, since it is unlikely that all the assets on your website are in different domains. In addition, many different caching tools are embedded in browsers that provide faster loading of content than shown in my diagram.

With caching

(30 image files x 50 ms to download each file) + (100 ms to perform one DNS lookup followed by caching) = 1600 ms

No caching

(30 image files x 50 ms per download of each file) + (30 x 100 ms for each DNS search) = 3000 ms

Why aren't my DNS records updated ?

There are other factors that increase the propagation time of changes. Some of them are listed below.

• Web browsers self-cache DNS records and store them for some time without TTL, which allegedly increases their speed. For example, modern versions of Internet Explorer by default cache DNS records for 30 minutes (up to IE 4 this time was 24 hours) and ignore lower TTL values.
• Mobile Internet providers may try to reduce total traffic by increasing TTL time, which reduces the frequency of requests.
• Complex internal networks with a larger number of DNS servers than expected have been updated longer for obvious reasons.

That is why in many services you can meet the following statement: "The full dissemination of changes in DNS records can take several days, so plan your actions accordingly."

Is it possible to somehow force the client to remotely update the DNS record?

This question is usually asked in the following context: “After updating the DNS records, the client cannot access some sites. How to perform the update force? "
Unfortunately, the only answer to this question: "None." There is no command in the DNS system that allows you to force an early update of data for lower level clients.

You can use commands to remove DNS records from the local cache, but they usually do not work as effectively as we would like, due to the presence of both upstream (caching DNS records on the side of the Internet provider) and downstream (caching DNS records in the browser) channels .

It is best to change the TTL values ​​in your records in advance.

3. RECOMMENDATIONS ON MANAGING CHANGES IN DNS RECORDS

Which TTL values ​​are better: small or large?

Developers have been waging a holy war for a long time about how to indent code in the code: using tabs or spaces. I found out that network administrators have roughly the same feelings when it comes to the duration of a TTL.
Typically, this view is supported by their own experience in repelling previous network attacks and troubleshooting network configuration problems.

A DDOS attack that can stop the work of the root DNS servers or similar servers of the Internet provider for 12 hours will have less impact on sites with a very high TTL value. In such cases, clients will work even if the DNS server is shut down or overloaded.
However, if you accidentally make a mistake when switching Internet or e-mail sites, 12 hours without any possibility of eliminating it is the last thing you need. That is why some administrators believe that the lifetime should not exceed 1 minute.

Personally, I try to specify a small TTL value for DNS records (less than 1 hour / 3600 seconds).

How do I know when a client requests an updated DNS record?

Determining when all customers update data is very difficult.
Lifetime is * not * an expiration date. You should not compare the TTL value in the DNS record with the recommended date of use, indicated, for example, on stale bread: this is not a definite time, at the occurrence of which the record will become invalid and will require replacement.

A DNS record is rather a staffing table, with changes slowly propagating throughout the network. When the clients in the “lower” schedule expire on the cache, they request a higher-level DNS server for the entry.
What is the best way to change DNS records?

Creating a “plan” or “strategy” to perform relatively simple tasks, which include changing a single record in a domain, may seem overwhelming, but given the enormous impact of DNS failures on the availability of your data, it is still worth some caution. As the old saying goes: “Preventing is easier than curing.”
There is an easy way to minimize errors: never update the DNS record and TTL value for this record at the same time. Ideally, you should have the following process implemented.

1. A few days before switching, set the TTL DNS record to a low value, for example, 300 seconds.
2. Set the switching date for the entry.
3. A few days after switching, set a higher TTL value.
What is the best way to add new DNS records?
Adding a new entry is easier than changing an existing one.
1. Add an entry with a low TTL value.
2. Check if everything works and increase the TTL value.

What is the most common TTL value?

Opinions on the * correct * TTL values ​​are so divergent that we attempted to determine it based on statistics. The list of the 500 most important Moz websites seemed to us an excellent cut of the Internet. In addition, a ready-made CSV file with a list of sites included in the final list was available on this resource.

I wrote a small script to iterate through the list and search for the current TTL value for the main entry in each domain. As with any data analysis project, this data will vary greatly depending on the question. The example is not comprehensive, it presents current (cached) results, etc., etc. Despite all these reservations, the results obtained still have a certain value.

Analysis of TTL values ​​for the 500 most important Internet domains according to Moz

You can view or edit this script or download it and perform the analysis yourself: gist.github.com/mbuckbee/79b2e76bd9271bea38487defd8a9138b
View the list and download it in CSV format at moz.com/top500

The minimum TTL value: 1
The maximum value of TTL: 129 540
Established Compliance Domains: 485
Arithmetic average TTL: 6468
Median TTL value: 300

Minimum values ​​were obtained from domains that very often change DNS records for load balancing. The maximum values ​​corresponded to domains that have not been updated for a long time (yes, python.org, I mean you).

If you need to justify your decision to set a low (within 1 hour, 3600 seconds) TTL value, you can provide a median value of 300 seconds (5 minutes) and confidently state that you have empirical evidence of your choice.

4. DNS PLATFORM TOOLS

How to check TTL value for DNS record in Windows?

The easiest way to check DNS records in Windows is to use the nslookup utility: technet.microsoft.com/en-us/library/bb490950.aspx .
Example: C: \> nslookup-type = cname -debug www.varonis.com


The TTL value is indicated at the bottom of the output. The phrase Non-authoritative answer indicates the TTL value received from the client (2 minutes 11 seconds before the local client checks the next level in the DNS chain).

How to check TTL value for DNS record in Unix / Linux / Mac?

On a Unix system (and its derivatives), the dig command is used to troubleshoot DNS problems.
Example: dig www.varonis.com


The TTL value is circled in red.

How to check the DNS record over the Internet?

Sometimes it happens that you need to check the DNS record without a computer at hand. A convenient (and free) version of the dig command is available in the Google tools at the following address: toolbox.googleapps.com/apps/dig .


The TTL value is circled in red.

How to ensure TTL distribution for DNS record?

If you need to find out whether the DNS record settings are updated on a specific DNS server, then in any DNS tool (dig, nslookup, etc.), instead of the local default setting, you can select the DNS server that will be used for the query.

For a complete picture of the changes, I recommend the whatsmydns.net resource, which allows you to check a lot of top-level DNS servers (the Internet provider level) and identify potential problems.



FUTURE ACTIONS TO SETUP A TTL FOR DNS RECORDS

Setting up a TTL for DNS records can be a daunting task, but if you select small values ​​(less than one hour), you can keep the network working and better prepare it for the changes.

If you enjoyed this article, I also recommend that you familiarize yourself with our course on the basics of web security. This will help you better protect the site or application for which you have just configured a DNS record. The course is free and very informative.