CloudFlare + nginx, or save with the help of the "coffee maker" (upd2: fast speakers dynamics!)

Good Friday, dear% username%, greedy reader and fighter for justice on the Internet!

We all remember (Google remembers!) That there was such an article CloudFlare + nginx = we cache everything on a free plan . In which the basic principles of saving on tariffs and servers were considered, by omnivorous caching on the side of CloudFlare files up to 512MB.

In this article we will play with the response codes of our server to save even more. gold to build a ziggurat and do not switch to the “enterprise plan” that “offer” us a similar result in our “offers”.

The free TTL tariff is 2 hours. This is what they have written in the free plan. If we dig deeper , we get:
')

• 200 301 120m;
• 302 303 20m;
• 403 1m;
• 404 5m;
• any 0s;

More deployed: redirects are cached for 20 minutes, authorization errors for a minute and classic 404 for 5 minutes. Other codes are not cached.

This means that if you have a cool dynamic website that, for example, provides an API for some operations, etc. etc. and you have it wrapped through CF, because it generates files and pictures and a lot of traffic, and the dynamics hurt ... then if someone understands how answers are generated (bare counter in base64 or your base or hashes, but the latter are harder to find ), then he (a person from the dark side of the room without lighting) can deceive other users with a false answer, namely 404.

For example:

• A simple service that pulls EXIF ​​from photos that have been uploaded. Link example http://example.com/api/image/exif/1.txt - which means to give EXIF ​​as TXT from a photo with id = 1.

Suppose you have 1336 photos there. This means that 1, 2..1336th will give out the 200th answer, which will be cached on the side of the cloudflare.

Solution # 2, without a crutch with 418 code (a consequence of this comment).

Solution # 1, crutch through 418 "i'm teapot" code.

Here is our emulation (nginx):

 location ~* "^/api/image/exif/(.*?)\.txt$" { default_type text/plain; set $testimageid $1; ##backend if ($testimageid ~ 1337){ return 404 "Image $testimageid not found"; } ##backend return 200 "EXIF for $testimageid image"; } 

And so on…



Until we reach the photo with Aidi 1337, which is not on the server yet. What will our dynamics do? It is logical that it will give some text about the error and the code 404.


Which will be cached next . And all subsequent requests to this URL will issue a response from the CF cache for the next 5 minutes .

Is it critical? Yes and no. It depends on how many users you make unhappy and how many millions on the CDN (thanks to me) you will lose millions.

What to do? We look at the official material and understand that it is necessary to change the code ... the server response code in the case of 404.

But not everything is so simple. It is impossible to read_account_and_to change_code_re answer.jpg , since each code carries some semantic load, which will be interpreted or simply hammered into it!

For this purpose, a code was invented that does not carry any semantic load, this is code 418, which is described in rfc2324 .

Change our emulation code (and clear the cache for this link in the CF panel):

 location ~* "^/api/image/exif/(.*?)\.txt$" { default_type text/plain; set $testimageid $1; ##backend if ($testimageid ~ 1337){ return 418 "Image $testimageid not found"; } ##backend return 200 "EXIF for $testimageid image"; } 

And we get the following picture:



This signifies the fact that as long as there are no pictures, CF does not cache the response from our server, sending a fresh response to clients each time with an error 418 [from our server] . Following, as the picture appears on the server, and our dynamics gives the answer 200, CF caches this answer for 2 hours.

Code 418 refers to the 4xx group of codes. And even if the end user does not have the definition of this code in the browser (or in the self-written software), then there is at least (must be!) A check on 4x as in the screenshots of this article (firefox, red square).

This is how you can simply report an error (4xx) and ask cloudflare not to cache this response with the policy when we are all caching. This saves from false 404 responses from the server when someone requested future material before the publication.

upd2: Solution # 2, without a crutch with 418 code. Corollary from this comment.

In the beginning we set the condition for Browser Cache Expiration:


Then create this Page Rule:



The main point: the lack of Browser Cache TTL and the presence of Cache Level: Cache Everything !

Then, if you have a TTL for valid answers of 315360000, then you need to add

 add_header Cache-Control "public, max-age=315360000"; 

in location'y, without adding it in the server!
eg

 location / { add_header Cache-Control "public, max-age=315360000"; } 

Location with dynamics is:

 location ~* "^/api/image/jexif/(.*?)\.txt$" { ##backend proxy_pass http://localhost:8080/headerstest/headerstest.jsp?imgid=$1; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; ##backend } 

Following makes our dynamics give errors with code 404 with such a header (for example, JSP)

 response.setHeader("Cache-Control", "private, max-age=0"); 

As a result, Cloudflare on the above settings does not cache errors, but that's not all!

If you need frequent cache updates for 200 responses of your dynamics, you need to specify (again using the example of JSP)

 response.setHeader("Cache-Control", "public, max-age=20"); 

And thus you get a 20 second cache of answers to your speakers! What is less than on the rate of Enterprise!

In fact, while this works, you can completely remove the site on the cloudflare and hide the server IP (provided that records other than A / AAAA / CNAME / TXT are located on different ip).

May the force be with you!

ZY, set $testimageid $1; It is used because nginx cannot compare $ 1 in the if expression (but it can operate $ 1 in the config or in the content), for this you need to enter a variable, but YourChief showed a more beautiful solution .. For the second solution you need to remove
add_header from server and leave it in location so that the first does not “stick” nginx to the second (this is the miracle in the response “public, max-age = 20, public, max-age = 315360000”).
Chipped boilers the pictures are aligned with each other and compressed in png- 8 , perfectionists on the dial-up can smile!