Improvements in the caching subsystem of Internet Explorer 9

The network plays a key role in the overall performance of a web browser. The best way to increase network performance is to reduce the amount of traffic transmitted over the network using HTTP compression and browser caching.

We have made a huge amount of improvements in how Internet Explorer 9 caches data in order to increase the use of resources from the cache. This post describes these improvements that are already available in the third version of the IE9 Platform Preview , which was released last month .

The essence of caching

Let's start with a quick description of how browser caching works. At the highest level, browsers perform two types of HTTP (S) requests — conditional requests and unconditional requests.
')
Unconditional requests are created when the browser does not have a cached copy of the resource. In this case, it is expected that the server will return a resource with an HTTP / 200 OK response . If the response headers do not specifically prohibit this, the client may cache the content and use it in the future.

If in the future the browser needs a resource that is in the local cache, then the resource headers are checked to see if the resource remains relevant. If the cached copy is up-to-date, no requests to the resource on the network will be made and the client simply uses the cached copy.

If the cached resource is outdated (older than the max-age header or later than the Expires date), the client will make a conditional request to the server to determine if the previously cached resource is relevant and can be used. A conditional request contains If-Modified-Since and / or If-None-Match headers that show the server what version of content is already on the client. The server can determine that the client version is still up to date by returning an empty response with the HTTP / 304 Not Modified header, or it can indicate to the client that its version is out of date by returning the response with the HTTP / 200 OK header and new content.

Obviously, conditional requests lead to better performance compared to unconditional requests (because the server does not need to send a resource if the client already has an up-to-date version), but better performance is achieved when the client knows which version in its cache is relevant and the conditional revalidation is not performed .

Headers with extremely long life spans

Although RFC2616 recommends that servers limit the lifespan of content to one year, some servers specify Cache-Control directives with a significantly longer period. Prior to IE9, Internet Explorer recognized as obsolete all resources whose lifetime ( Cache-Control: max-age value) was more than 2147483648 (2 ^ 31) seconds, approximately 68 years.

In the ninth version of IE, we accept any set max-age up to 2 ^ 63, although inside we reduce it to 2 ^ 31 seconds.

Vary parameter improvements

The HTTP / 1.1 Vary response header allows the server to indicate that the actual cached resource can be used without revalidation only if the headers specified in Vary match the request headers.

For example, this allows the server to return content in English with the heading Vary: Accept-Language. If the user later changes the Accept-Language value from en-US to ja-JP in his browser, the previously cached content will not be used, since the Accept-Language header no longer matches the value that was saved when caching the resource in English.

In IE9, we have improved support for key Vary usage scenarios. So, IE9 no longer requires server revalidation for responses that contain the Vary directive : Accept-Encoding and Vary: Host.

We can safely support these two directives for the following reasons:

1. all requests are explicitly dependent on Host, because the host is a component of the request URL;
2. IE always unpacks HTTP responses into a cache, which makes Vary: Accept-Encoding redundant.

Like IE6 and higher, IE9 also ignores the Vary: User-Agent directive .

If the response contains a Vary directive that specifies a header or a combination of headers other than Accept-Encoding , Host, or User-Agent , then Internet Explorer will continue to cache the response if the response contains an ETAG header. However, such a response will be marked as obsolete and a conditional HTTP request will be made before its use in order to determine the relevance of the cached copy.

Caching redirects

IE9 now supports caching responses with HTTP redirects, as described in RFC2616. Answers with permanent redirect (with status 301) will be cached until they contain headers that prohibit it (for example, Cache-Control: no-cache ) and those requests that have a temporary redirect status (302 or 207) will also be cached if the headers allow it (for example, Cache-Control: max-age = 120 ).

Despite significant improvements in performance, web applications that are mis-configured may not work as expected after these improvements. For example, we found several public sites that use templates similar to the following:

> GET / HTTP / 1.1

<301 Redirect to / SetCookie.asp

> GET /SetCookie.asp HTTP / 1.1

<301 Redirect to /

Here, the purpose of the site is to set a cookie using a helper page if it is not set. The problem is that the server chose 301 for this task and 301 is cached. As a result, IE will simply perform redirects on the client side without accessing the server until the user is bored and he closes the browser. Please note that this infinite loop will be also if cookies are disabled in the user's browser.

If your site creates redirects, you must make sure that they are configured to avoid looping and verify that specific redirects (for example, to set cookies) are marked as non-cacheable.

HTTPS caching enhancements

A few months ago, I mentioned that IE would not reuse previously cached resources obtained via HTTPS until at least one secure connection was established for the current process. This can lead to the fact that previously saved resources will be ignored, which will lead to unconditional network requests for content, although it is already in the cache.

In IE9, optional HTTPS cross-host requests will now be conditional, so the server can simply return an HTTP / 304 Not Modified response for unchanged content. Although there is still a need for a request, a significant increase in performance can be achieved since the server will not need to transfer the resource to the client.

Optimization of functions Forward / Back

In IE9, we made a number of improvements so that the “Forward” and “Back” buttons lead to a quick load.

RFC2616 specifically indicates that the forward / backward mechanism of browsers should not be cached:

History and caching mechanisms are different mechanisms. In practice, history mechanisms should not attempt to show the current state of a resource. On the contrary, the mechanisms of history should show exactly what the user saw during the time when he visited the resource.

By default, the expiration time of the resource’s relevance should not be relevant to the history mechanisms. If an entity remains in the repository, the history mechanism must display it, even if the entity is expired, unless the user specifically configures the agent to update expired documents.

In previous versions of IE, when a user went forward or backward, IE checked for resource updates if they were sent with the must-revallidate cache directive and depending on some other parameters of how the resource was obtained. In IE9, the INTERNET_FLAG_FWD_BACK flag is used and IE will not check the relevance of cached resources when the user goes forward / backward.

As a result of this optimization, Internet Explorer 9 can perform significantly fewer conditional HTTP requests while the functions are running forward / backward. For example, the following table shows the increase in performance compared to previous behavior:

-	IE8	IE9	Improvement
Navigation forward / backward	Number of requests: 21 Bytes are transferred: 12,475 Bytes received: 216,580	Number of requests: one Bytes are transferred: 325 Bytes received: 144,617	Number of requests: -20 (-95%) Bytes are transferred: -12,150 (-97.4%) Bytes received: -71,963 (-33.3%)

After I talked about the fact that we ignore caching directives while navigating forward / backward, attentive readers may ask why IE9 still produces one request when the user presses the back button. The reason is that IE does not send any cached resources to the cache. Resources that are prohibited for caching are delivered with the Cache-Control directive : no-cache or with the Expires date value in the past or when the Expires date is not later than the Date header value. For this reason, the browser has to reload such resources when the user goes forward / backward. To increase performance and create the ability to load a resource into the cache while navigating forward / backward, simply replace Cache-Control: no-cache with Cache-Control: max-age = 0, in this case, revalidation will occur in all cases except forward / backward navigation .

Unlike other improvements that we have described, the improvement in the “forward / backward” operation cannot be observed in the preview version of Internet Explorer 9 Platform Preview, since it does not have “forward / backward” buttons .

Improvements in caching heuristics

According to the recommendations, web developers should explicitly indicate the expiration time of their resource in order for browsers to reuse resources without having to make conditional HTTP requests for revalidation to the server. However, many sites deliver content without information about the expiration of relevance, leaving the browser to care about the relevance of the resource.

IE allows users to configure what should happen to content that is received without information about the expiration of relevance. In the Tools> Internet Options> Browsing history> Settings settings, you can find four options:



These options offer the following behavior:

Every time I visit the webpage	Resources with no explicit time information of relevance are marked as outdated and will be reloaded with the server before each use.
Every time I start Internet Explorer	Any resource without explicit information about the time of relevance validates with the server at least once per session (and every 12 hours in this session)
Automatically (Default)	IE will use the heuristics mechanism to determine relevance.
Never	Any cached resource will be flagged as relevant and will not be reevaluated.

These options control the behavior of the browser only when the content is received without information about the period of relevance. If the content is specified with an explicit rule (for example, Cache-Control: max-age = 3600 or Cache-Control: no-cache ), then the browser will use these server rules and these options will have no effect.

In previous versions of IE, automatic heuristics were simple and only affected cached images , but IE9 made improvements in heuristics according to RFC2616:

If the answer contains the Last-Modified value, then the heuristic value of the time of relevance must not be greater than the interval from this value of time. A typical value should be 10% of this interval.

If IE9 obtains a resource that can be cached without a fixed relevance lifetime value, the heuristic validity period is calculated as follows:

max-age = (DownloadTime - LastModified) * 0.1

If the Last-Modified header is not in the server's response, then IE will apply the “Once per browser session” revalidation behavior (once per session).

As a result of improved heuristic caching, IE9 can produce fewer conditional HTTP requests when reloading most pages. For example, the following table shows the improvement obtained when visiting a regular article on a popular site:

-	IE8	IE9	Improvement
Re-visit to new browser session (PLT2)	Number of requests: 42 Bytes are transferred: 26050 Bytes received: 220681	Number of requests: 2 Bytes are transferred: 1134 Bytes received: 145217	Number of requests: -40 (-95.3%) Bytes are transferred: -24916 (-95.6%) Bytes received: -75464 (-34.2%)

The caching inspector in Fiddler will show you when the answer loses relevance based on the response headers. For example, below is what you see in the response, which contains the ETAG and the Last-Modified header, but does not contain information about the relevance period:

Other network improvements

In this post, I have listed the improvements in the Internet Explorer caching code that help make sure the best use of the network by websites. Of course, web developers should continue to follow the recommendations and specify the behavior for caching using the Expires and Cache-Control headers, but even if sites don't do this, they will still load much faster in IE9.

In the following articles, I will describe the other improvements that we made in the IE9 network stack for even faster loading pages.