What protect sites, or why do we need WAF?

This year, Positive Technologies was named the “visionary” in the Gartner Magic Quadrant for Web Application Firewalls rating. This raised a number of questions about what achievements we got there and what WAF is all about. The questions are quite legitimate, since Gartner has been releasing its WAF research only since last year (for example: the "quadrants" on SIEM began to appear five years earlier, in 2009). In addition, some are still confused with the terminology, not distinguishing between a “web application protection screen” (WAF) and a regular network firewall or an intrusion prevention system (IPS).

In this article we will try to separate flies from cutlets - and tell you how the perimeter defense is evolving as the sophistication of attacks increases.

')

1. At the beginning of time: packet filters

Initially, the term firewall (firewall, screen) refers to a network filter that is placed between the trusted internal network and the external Internet (hence the adjective "internetwork"). This filter was designed to block suspicious network packets based on the criteria of the low levels of the OSI model: at the network and data link levels. In other words, the filter took into account only the source and destination IP addresses, the fragmentation flag, and the port numbers.

Further, the possibilities were extended to session level gateways, or stateful firewall state channel filters. These second-generation firewalls have improved the quality and filtering performance by monitoring packet belonging to active TCP sessions.

Unfortunately, this protection system is practically useless against modern cyber threats, where more than 80% of attacks exploit application vulnerabilities, rather than network architecture and services. Worse, blocking certain ports, addresses or protocols (the main way firewalls work) can cut down quite legitimate applications. This means that the protection system should conduct a more in-depth analysis of the contents of the packets — that is, it is better to “understand” the operation of the applications.

2. Intrusion Detection / Prevention Systems (IDS / IPS)

The next generation of protective screens are intrusion detection and prevention systems (IDS / IPS). They are capable of examining data fields in TCP packets and inspecting at the application level for certain signatures. IDS systems are adapted to detect attacks not only outside, but also inside the network by listening to the switch's SPAN port.

To improve the protection mechanisms in IDS / IPS, decoders (parsing TCP packet fields) and preprocessors (parsing application protocol parts, for example, HTTP) have been used. The use of preprocessors in IPS Snort made it possible to significantly improve the functionality of perimeter protection in comparison with a packet filter, even if the latter checks packets at the application level (iptables with the layer7 module).

However, the main drawback of the packet filter remained: the check is carried out per package, without taking into account sessions, cookies, and the rest of the application’s logic.

In parallel, proxy servers appear to combat the spread of viruses, and reverse proxy servers are used to solve load balancing problems. They differ technically, but the main thing is that both of them fully work at the application level: two TCP connections from proxy to client and from proxy to server open, traffic analysis is conducted exclusively at the application level.

3. All in a bunch: NGFW / UTM

The next step in the evolution of intrusion detection systems was the emergence of UTM (unified threat management, system of unified threat management) and NGFW (next generation firewall, new generation screens) devices.

UTM systems differ from NGFW only in marketing, and their functionality practically coincides. Both classes of software products were an attempt to combine the functions of different products (antivirus, IDS / IPS, packet filter, VPN gateway, router, balancer, etc.) in one device. At the same time, the detection of attacks in UTM / NGFW devices is often carried out on the old technological base, using the above-mentioned preprocessors.

The specificity of web applications suggests that during a single user session with a web server, a large number of different TCP connections can be made that open from different addresses, but have one (possibly dynamic) session identifier. This leads to the fact that a platform based on a full-featured reverse proxy server is required to accurately protect web traffic.

But the difference in the technology platform is not the only thing that distinguishes the protection of web applications.

4. Protecting the Web: What WAF Should Know

Speaking quite simply, web applications differ from ordinary applications in two things: a huge variety and significant interactivity. This creates a number of new threats that traditional firewalls do not cope with: according to our estimates, in 2014, 60% of attacks against corporate networks were carried out through web applications, despite the presence of traditional defenses.

This is where the Web Application Firewall (WAF) comes in, a firewall for applications that transmit data over HTTP and HTTPS. Here are the features that distinguish WAF from previous generation security systems:

	Waf	IPS	NGFW / UTM
Multiprotocol Security	-	+	+
IP Reputation	±	±	±
Signatures attacks	+	±	±
Automatic training, behavioral analysis	+	-	-
Protection users	+	-	-
Scanner vulnerabilities	+	-	-
Virtual patching	+	-	-
Correlations, attack chains	+	-	-

Now more about what all these points mean:

Multiprotocol Security

Being a highly specialized tool, WAF does not protect against protocol problems other than HTTP / HTTPS. But any medal has two sides. The variety of ways to exchange data over the HTTP protocol is so great that only a specialized tool can navigate it. Just for example: somewhere, variables and values ​​are transmitted in the format of example.com/animals?dogs=32&cats=23 , somewhere in the format of example.com/animals/dogs/32/cats/23 , somewhere the transfer of application parameters carried out in a cookie, and somewhere in the parameters of the HTTP header.

In addition, advanced WAF models can analyze XML, JSON, and other protocols of modern portals and mobile applications. In particular, this allows counteracting most protective screen bypass methods (HPC, HPP, Verb Tampering, etc.).

IP Reputation

IP-Reputation technology relies on external black and white lists of resources, and is equally accessible to any perimeter protection. But the value of this method is somewhat exaggerated. Thus, in the practice of our specialists, there were situations when large news agencies, because of their vulnerabilities, distributed malware to users for months, and at the same time did not fall into the black lists. Unfortunately, malware penetration vectors are very diverse, and nowadays even sites of public authorities can be a source of infection for users. And the reverse problem is possible when “innocent” resources are blocked by IP.

Attack signatures

The signature approach to detecting attacks is applied everywhere, but only competent traffic preprocessing available for WAF can ensure adequate signatures. Disadvantages of preprocessing lead to excessive “monstrousness” of attack signatures: administrators cannot understand the most complicated regular expressions, the whole point of which is that their authors, for example, just tried to take into account the possibility of passing a parameter both in plain text and in the form of hexadecimal code with by percentage.

Automated learning and behavioral analysis

Attackers actively use zero-day (0-day) vulnerabilities to attack web applications, which makes signature analysis methods useless. Instead, you need to analyze network traffic and system logs to create a model for the normal functioning of the application, and on the basis of this model to identify the anomalous behavior of the system. WAF, by virtue of its architecture, can parse the entire user session, and therefore is capable of more in-depth behavioral analysis than NGFW. In particular, it allows detecting attacks using automated tools (scanning, password selection, DDoS, fraud, involvement in botnets).

However, in most cases, the behavioral model training consists in the fact that operators from somewhere take “white traffic” and “feed” it to their means of protection. But after commissioning, user behavior may change: programmers finish the interface according to the adjusted technical specifications, designers “add beauty”, advertising campaigns change the direction of attention. It is impossible to chart once and for all the behavior of the “right” visitor. At the same time, only units of software products can be trained on real, “gray” traffic - and these are only WAFs.

User protection

The perimeter equipment discussed in this article is designed to protect servers with web applications. However, there is a class of attacks (for example, CSRF) aimed at the client of the web application. Since the attack traffic does not pass through the protective perimeter, at first glance it is impossible to protect the user.

Consider the following attack scenario: the user enters the bank site, authenticates there, and then opens the infected resource in another browser tab. JavaScript, loaded in another window, can make a request to transfer money secretly from the user, and the browser at the same time substitute all the necessary authentication parameters for the implementation of a financial transaction, since the user's communication session with the bank has not yet ended. In this situation, there is a weakness in the authentication algorithm in the banking software. If for each form contained on the site page a unique token was generated, there would be no problem.

Unfortunately, software developers almost never do this. However, some WAFs can independently implement such protection in web forms and thus protect the client — or rather, its requests, data, URLs, and cookies.

Interaction with vulnerability scanners

Perimeter equipment is assigned not only the task of protecting web applications, but also the task of monitoring attacks. At the same time, competent monitoring is based on an understanding of the weaknesses of the protected software, which makes it possible to weed out irrelevant attempts at attacks and isolate only those relating to real vulnerabilities present in the system.

The best samples of WAF are available with integrated vulnerability scanners operating in black box mode or dynamic analysis (DAST). Such a scanner can be used in real time to quickly check for vulnerabilities that intruders attack.

Virtual patching

Even known vulnerabilities cannot be fixed right away: fixing a code requires time and money, and often, stopping important business processes; sometimes, when using third-party software, fixing is not possible at all. For countering such "private" threats in IDS / IPS systems, and inherited from UTM / NGFW, custom signatures are used. But the problem is that writing such a signature requires a deep understanding of the attack mechanism from the user. Otherwise, the user signature can not only “miss” the threat, but also generate a large number of false positives.

In the most modern WAF, an automated approach to virtual patching is used. For this, an application source code analyzer (SAST, IAST) is used, which not only shows the lines of the vulnerable code in the report, but immediately generates an exploit, that is, a call with specific values ​​to exploit the detected vulnerability. These exploits are sent to WAF to automatically create virtual patches that provide an immediate “closing in the gap” before the code is fixed.

Correlations and attack chains

A traditional firewall provides thousands of suspicious events that need to be manually handled to identify the real threat. As noted by Gartner, vendors of IPS systems generally prefer to disable most web application signatures to reduce the risk of such problems.

Modern WAF can group similar operations and identify a chain of attack development - from intelligence to theft of important data or bookmarking. As a result, instead of a list of thousands of suspicious events, information security specialists receive several dozen truly important messages.

What's next?

It is clear that the solutions of different vendors of WAF will always differ in the set of functions. Therefore, we list here only the most well-known additional features of modern protective screens of the application level:

• Work with SSL-traffic as an additional level of protection. According to Gartner analysts, the ability to check encrypted traffic is one of the most important differences between WAF and regular firewalls and IPS.
• Authentication services: WAF is a single entry point for web applications or acts as an authentication broker for legacy applications whose authentication mechanism is broken.
• Support Content Security Policy (CSP) to protect against XSS and other attacks.

In addition, experts from Positive Technologies predict the following promising directions for the development of application-level firewalls in the near future:

• New behavioral analysis algorithms that will better distinguish users to identify bots and intruders (UBA).
• Protection of applications built on the basis of HTML5, protocols based on XML, using non-relational databases (NoSQL).
• WAF, focused on specific types of applications: banking (RBS), ERP systems, telecom applications, mass media, etc.

Finally it is worth noting that this article is devoted only to the technological features of WAF. But in practice it is worth taking into account organizational ones - for example, compliance with security standards or the possibility of integrating WAF with other security tools (antivirus, DLP, etc.). Deployment models can also be different: it can be a hardware, software or virtual solution, or a cloud service in SaaS, VAS and MSS models.

PS We remind you that you can get for free testing a protective screen of the PT Application Firewall level here: af.ptsecurity.ru