Fighting DDoS eyes Highload Lab
Alexander Lyamin, the creator of Highload Lab and QRATOR traffic filtering network, tells about trends and trends in DDoS attacks. We took the interview at the beginning of the year, but little has changed so far.
DDoS attacks appeared at the dawn of the Internet. My personal acquaintance with them began during the IT Territory project in 2003, when the game was just launched. She had a fairly aggressive advertising campaign, in response to which DDoS immediately flew in from competitors. Frankly, I was confused. Most of all due to the fact that the company that provided hosting services, not only could not, and did not want to fight the attack. Her representatives said that it was not their problem.
Literally overnight, we rethought the structure of the application and rebuilt it. The resource rose, but the attack stopped working at the application level. The attackers transferred the attack to the network level, as a result of which the entire network of the hosting company was disconnected. Then she came to us with a request to do something. The question “what to do?” Was followed by the answer: “Well, agree somehow, you know who is attacking you.” Naturally, we did not know and could not agree. And how to negotiate with the terrorists?
')
The company, which, it would seem, should take care of customers, did nothing. And not because she is bad and poor-quality hosting, but because she could not do anything.
The most popular DDoS attacks are, of course, attacks organized with the help of botnets. This is an affordable way to make the attack distributed.
The cost of an attack strongly depends on how it is implemented. The performer may be a student who wrote something himself and is ready to work for a beer, or maybe an organized group.
There are types of attacks that can cost, according to rumors, half a million rubles and more. We distinguish attacks of a basic type: up to five thousand bots, conducting at the application level, one strategy. For the performer there is nothing complicated - got WMZ, pressed the button, went to drink beer. It costs about $ 30–100 per day. But there are other kinds of attacks, when it is clear that the team is working, and it works 24/7 - for the result. If she fails to achieve a result, she constantly switches attack modes, changes her strategy, tries to find a vulnerable spot and break through. Of course, this is not far $ 100 per day.
Russia stands out among other countries by far more sophisticated attacks. Europeans are shocked by how complex our attacks are. For example, we have recently been approached by one company that operates on the Russian market, but all of its information structures are located in one of the leading European data centers. When the company approached us, the data center experienced serious problems and was unavailable. We prepared for the reception of the company on our network and warmed up the quarantine equipment, waiting for something extraordinary, because the data center was "dead"! Imagine our surprise when we saw that, although the attack was carried out at a level above average, it does not constitute anything special.
Legislation is extremely weak in matters of accountability for DDoS, which is why attackers feel that they are unpunished. For a qualified programmer, conducting DDoS attacks on an order becomes an absolutely safe profitable business, the income from which can exceed current salaries in the market tenfold.
Motives for DDoS are, as a rule, money and just personal dislike. De facto we live in an information society. The speed at which information is distributed affects it in a direct way. Blocking the source of information, you can irreversibly affect society. Accordingly, DDoS is an effective means of blocking any source of information for the required time.
An example is the site Slon.ru. The site worked, everything was loaded, but the attack did not fall. Such attacks are called combined. When they came to us, it was conducted at the network level, on the runway. When the attackers saw that filling the bar did not work at all, the application-level attack began. The botnet used for the attack included about 200–270 thousand bots.
The most subject to DDoS attacks are narrow enough areas of highly competitive Internet businesses. A good example is the pirated clones of Lineage II. Such services are a separate story, because they are purely commercial. If at two o'clock in the morning someone knocks to you in ICQ and, having allowed eight spelling errors in four sentences, requires (!) That you should help him immediately, you can be sure that this is him! Administrator Pirate Lineage!
DDoS-attacks are also those sites where it is difficult to expect them. We, for example, have such an internal meme - “cedar barrels”. An online store that, believe me, sells cedar barrels, has undergone a serious DDoS attack. This is a very narrow, highly competitive type of activity, which apparently brings a good profit.
Why do I need to classify attacks? To understand, sort through their work mechanism and take adequate countermeasures.
Our colleagues are trying to somehow classify the attack. On the same site you can find ICMP spoof, DNS amplification, and TCP SYN flood, TCP RST flood - the guys list the attack techniques. A lot of scary letters that for the average user do not carry any meaning. This classification does not suit us.
We classify attacks very simply: attacks on applications, attacks on a channel strip (speed is measured in gigabits / s), attacks on a network infrastructure (speed is measured in packets per second), attacks on a transport layer (TCP / IP stack).
The flesh itself is the application level. Why? Because the application level attacks have maximum leverage. The attack shoulder is the ratio of the resources needed on the side of the attackers to the resources needed on the application side (on the side of the defenders).
Take some average online store. You can find a link, a regular link, a certain number of hits per second to which this store will kill completely. There are many such applications, and in order to kill them, sometimes a botnet is not needed - a cell phone is enough not even with EDGE, but with GPRS. Four to five requests per second, and the application is thrown into the outflow and cannot exit from there until the server is restarted. This is due to the popularity of attacks at the application level.
There is also a transport layer - attacks aimed at the TCP / IP stack itself. This type includes attacks such as SYN-flood, RST-flood or FIN-way - a trendy attack now with incorrect connection closure, which, by the way, also exploits the vulnerability of the protocol specification, and not the implementation.
The most popular techniques include DNS amplification. It is enough to find any IDP based network service without handshake, send an N-size packet with a falsified source and receive an N x K packet in response. In this case, to implement a distributed attack, you need to have a list of IP addresses that have these services, one very well connected server on gigabit, which will give a set of packets to these "reflectors" with the shared IP address of the victim. The packet will go to the victim and reduce its throughput to zero and take it out of working condition. This is a DNS server - UDP 53, where you can do such a thing as a recursive query on a zone. By itself, it is small, but the answer to it will be long. To further increase the K-attack coefficient, it is enough to “feed” these servers with some large fake domain zones. Getting them recursively, with the fake address of the victim, you can increase K at times. The second option is NTP, time synchronization protocol, which also has similar vulnerabilities in poorly configured servers.
A SYN-flood attack appeared simultaneously with the TCP protocol. The first mention of her I met, in my opinion, in 1982. Oddly enough, it is effective to this day. The story develops in a spiral. One hundred and five hundred SYN requests per second is, of course, a stage long gone. Currently, with sufficient processing capacity, you can easily surpass the figure of 10 million packets per second.
At the time of sending a packet with a connection request, the partner must generate a sequence number (and this requires computation, since it must be a random, crypto-resistant number) and create a definite record on its stack. It requires resources, resources and resources again. Sending a packet by generating its traffic requires significantly less resources than actions performed on the server side. We get a shoulder attack.
The problem exists within the protocol itself, in its specification. When TCP / IP-stack was developed, no one thought that the Internet would grow to such a scale by the number of nodes, reach such speeds and, which is important, will pump so much money through itself.
There are attacks that do not use botnets. Distributed attack can be carried out using the mechanism of reflection. The classic of the genre is attacks of DNS amplification with reflection and power increase.
Attacks on the infrastructure affect everything that lies around the network infrastructure: routing protocols and the hardware itself, if the management modules have a public IP address.
What are attacks at the network level? Just fill the band at 56 Gbps - this is the clustery sort called. This is the last resort, when nothing helps. Such attacks are very expensive and extremely destructive, not only for the victim herself, but also for everyone who “stands nearby.” As a rule, they cannot last longer than two or three days, as they begin to cause problems even to the sources of attack - the networks from which it is produced.
Basic attacks made with the help of botnets, which have about 200 bots and do not know anything except “get root”, should not in principle be a problem for a competently written resource.
You can protect yourself from any attack. We do not doubt it.
Usually, when a DDoS attack starts on a client, the hoster doesn’t find anything smarter than just turning it off, as he is afraid that his other clients will fall down too.
When a client comes to us, we explain that he needs to transfer the DNS to our IP address (which we allocate to him). Also, in order to avoid an attack on a direct IP client that is already lit up on the network, it is necessary to change it at least, and as a maximum - change this IP and additionally hide all IP addresses except ours using iptables settings or a firewall.
As soon as the DNS is rebuilt, filtering begins. And then the most interesting thing happens - filter training. Usually we set the bar for ourselves: after two hours, under attack, the client’s resource should start working. And in general, we maintain it.
The security system of our Qrator service is based on a variety of mathematical constructions. As usual, Yandex responds to the question “how is your search arranged?”? Yes, just! We take the text, tokens, dividers, build indies, we rank. We have about the same thing, only we solve the problem of analyzing and filtering traffic. Her decision employs a lot of people.
Behavioral analysis is one of the most effective methods of filtering traffic. We view the site as a transition tree. There are pages in the tree nodes, and in the edges we lay the transition probability and the transition delay. It is based on the simple fact that robots and people see web pages differently. When there is enough time for training, people “tread out” in these transitions certain seals — paths. All visitors who drop out of them, with varying probability, are robots. It seems to be all simple. On the other hand, if you estimate how much memory and computational resources will be needed for processing, you will understand that, probably, with current computing power, this is not so easy.
If a client comes to us already under attack, learning from attack ... is not impossible, but difficult. Often this requires monitoring by the engineer. That is why for the connection under attack, we are forced to take some additional charge.
We recommend connecting to our network before a DDoS attack. Of course, we try to minimize the training time under attack. With us, it is not a week, like Cisco Guard (this is our iron rival, which has been discontinued), but only a few hours.
Anyone who says that he has a false zero is a charlatan. False alarms (when legitimate visitors are blacklisted) are unfortunately inevitable. If only because there is a proxy, there are NATs, there are just people who do not behave like regular users. A classic example is site administrators. The administrator can load the server as 30, 40 and even 100 users.
We had one complaint about Cisco Guard: when you connect the attacked service to it, then regardless of whether there is an engineer there or not, the first day the service works in such a way that it would be better if it didn’t work at all. From this it became clear that it is impossible to protect against DDoS attacks at the application level without understanding the semantics of the application protocol. Semantic analysis is required, as well as behavioral.
We clearly understand that there is no “silver bullet”: what will work well in some situations will not work in others. The Qrator classifier is a complex set of algorithms that make up a voting system. We try to develop and add tools to the toolkit and, I hope, we will find some more effective methods in the near future. Some ideas are already there.
Approximately one million dollars will cost a piece of hardware from Arbor, capable of cleaning 10 Gbit. Plus a person, plus channel capacities ... At the same time, we see attacks at speeds above 10 Gbit / s about once a month and a half.
We tend to distinguish two types of bands: active and passive. In the active band, you can terminate and analyze any TCP connection and make a decision on it. The passive band is the band for which control you need to set a bit mask for which traffic will be cut. Thus, something intellectual can not be cut there. If we talk about the active band, almost all of our traffic providers block UDP from a specific address, if necessary, all ICMP or ICMP on a specific signature. On this strip, we quietly lived 57 Gbps. We are sure that we can live more. Such attacks do not cause any special problems, except for the need to pay for this band, that is, we are talking about a figure of more than 100 Gbps for the passive band. As follows from the situation with DDoS attacks on the Russian market, this is quite enough.
The advantage of Qrator (as a service) over purchased Arbor is that our solution is not a point solution. The network is built on BGP-AnyCast, we choose to install points exclusively trunk operators. We do not put points on public exchange simply because it does not guarantee the quality of service. The network is developing thanks to our own modeling algorithms. We build it so that it is possible to distribute the load on the network elements more or less evenly.
Inside the point of presence, the system is also scalable. The point is not one “piece of iron”, there are several of them. There is a quarantine equipment on which some attacks "land".
We created a model that allows you to mathematically calculate how traffic is distributed over the Internet when certain BGP announcements appear. This allows us to develop harmoniously and build a network that can really be balanced across nodes.
We are not tied to a single telecom operator and we try to distribute risks across all operators with whom we work.
We tried for a long time to deal with the TCP / IP stack, looked at Free BSD and Linux, and eventually we came to the conclusion that we didn’t like the stack in its current state. We have our own lightweight version of TCP / IP, which behaves very well on current short-lived protocols, fast TCP connections.
We do not hide that the filtering node is running Linux. Linkus is a container in which the platform is managed and the mathematical transformations necessary for behavioral analysis are performed. A substantial part of the TCP stack lives in the TCP card itself, which is why, in fact, we have such good rates of speed / packet handling. One of our filtering sites is able to shovel 6 Gbps of traffic.
You can defend against basic attacks. To do this, you must have a dedicated hosting, as well as the ability to compile the modules and your version of the web server. There are a lot of articles on this topic, and I will probably refer you to my 2008 article (you can find it on the blog at highloadlab.ru). This is one of the first articles in which it is easy to explain what and how to do. I also recommend that you familiarize yourself with the presentation “A Practical Guide to Survival in DDoS”, which we showed at Highload ++ in 2009.
We tried to write articles on Habré and tell at an industry conference how to defend oneself against basic attacks. But, unfortunately, this had no effect.
DDoS is one of the ways to monetize a botnet, but far from the most profitable. There is also spam, fraud, advertising clips and so on.
I listed the botnet properties on the slide for one fairly old presentation. When I did that slide, it seemed to me absolutely correct:
A couple of years ago, we first saw a slow botnet that was not greedy ... He made one absolutely legitimate request every five minutes. We were surprised, but at the same time the botnet, which counted 75 thousand bots, still caused problems. Try filtering THIS.
Now, of all the points listed above, only one remains - the botnet’s desire to self-preserve. Botnets are no longer greedy, not stupid and not flawed. Now we are dealing with full-fledged minimized web browsers with Java scripts, redirects, cookies.
Distributing commands to members of the 20,000th botnet, given that the bot itself is the initiator of the connections, is not a trivial task. As a rule, control panels, to our surprise, are written on the same LAMP Stack (Linux, Apache HTTP Server, MySQL and PHP). Until 2010, the deployment of the five-thousandth botnet in the direction of the resource of the victim took 30-40 minutes.
In 2010, botnet management began to be organized using P2P. The guys began to simply distribute commands to the super-fast: within five to six minutes a botnet, numbering 10-20 thousand bots, can extend the team inside and turn around for a resource.
Botnets try to imitate user behavior as precisely as possible in order to make it more difficult for them to be detected and filtered, to isolate the botnet body and block it.
For example, not so long ago there was a surge in the activity of the MinerBot botnet, which produces BitCoin. He comes to cover pages without a referrer, randomly, follows links and really creates problems for solutions like Cisco and Arbor. They are not able to filter out MinerBot because it does not have any of the flaws that these solutions are aimed at detecting.
Botnets also ceased to be transnational - downloads are easily sold by region. The first time we saw this was in 2009, when a botnet for 1500 heads “came to us”, and everything was clean in the CIS.
"Falling asleep botnet" - so we call a rather fashionable attack. The botnet detects that it is all filtered, distributes the command and stops the attack centrally. After that, the random member of the botnet sends test requests waiting for the filter to turn off. As soon as it turns off, the attack resumes within three to five minutes. This is dangerous because such an attack can last forever - no resources, from the point of view of the bot, it does not consume.
Different botnets are very different from each other. The attack technique itself is constantly changing.
With millionaire botnets, there is a very interesting situation. In the past few years, the number of those who would like to acquire their botnet has increased significantly. A simple experiment: put Windows XP SP1 on an honest IP address. How long will he live before something lands on him, even if he doesn’t open the web browser? Maximum five minutes. There are many teams that are struggling to increase the body of a botnet, and the supply of vulnerable systems is extremely limited. Accordingly, the number of botnets is growing, but their size is slowly but surely decreasing. Botnets are already beginning to intersect, that is, one computer is a member of several botnets at once.
There are fewer and fewer botnets on tens of millions of computers. They have quite a Jedi. :)
We do not have the ability to do reverse-engineering of the botnet code, because we do not have the administrative capacity to seize his body and, most importantly, we do not have our own specialists capable of doing reverse engineering of the code oriented to Windows systems.
The idea of ​​practicing DDoS attacks originated in Moscow State University. We looked at how things are going with the resilience of government resources and the resilience of web applications in Russia as a whole. It became clear that our services are likely to be in demand. After all, a fallen online store is a problem only for its owner, but a fallen tax inspection is a problem for the whole country.
Start research - it was my personal initiative. The university provided the infrastructure, I bought the equipment with my own money.
In 2008, we had an idea. In 2009, a beta version of the product appeared, which we tested in open beta mode for most of 2010. We hosted any project in distress on our site. It became clear that we cope well with this task, even with a limited university infrastructure. We, for example, helped the Vedomosti newspaper. Was cool. :)
The need for commercialization pushed us: in June 2010, when the maximum capacity of the university network was 10 Gbps, an attack of 12.5 Gbps fell on us. The attack showed that the filters are coping, and we can easily overcome a more powerful attack, but channel capacities are needed. This is a valuable and expensive resource, but I also do not want to lose ... We had some of our accumulated funds, for which the channel capacities were purchased. Also, additional equipment was purchased.
We were lucky with the launch - we had a wonderful stress test. That is, on September 1, according to the plan, I just put the last entry point, and on September 2, Habrahabr came to us under the attack of 6 Gbps. We got a free stress test.
Traffic is one of our main expenses.It is spent not just a lot, but a lot.
The company works in several directions: we develop custom-made high-load web applications and advise on their creation. The second direction, the most promising for us, the most dynamically developing, is our “boxed” product, the Qrator traffic filtering system. In it we invest almost everything that we earn.
At the moment, our company employs 12 people. The non-technical staff includes eight engineers and four other employees. Two of them are freelancers from Moscow. At the beginning of the year, if everything goes well, we want to invite two more engineers to the company. Just like Yandex, we are looking for mathematicians who can program, work with data (structured and poorly structured).
Unfortunately, we are not engaged in reverse engineering, but we see that each attack has its own signature and logic.
We have been in existence for a year and a half. It was not an easy time. At some moments it was very difficult both financially and morally. But during this time we figured out what issues arise during the operation of the service, understood how to create a tariff stack. Since the service is new, no one knows how to sell it. All offers on the market have certain flaws.
Highload Lab is profitable. This year we made major technical changes - we developed a new version of our specialized network processors - and actively developed partnerships with all interested companies: hosting companies, telecoms.
One of our goals is to provide protection for small businesses. This is the most unprotected layer of attacks. Many companies take for protection from DDoS attacks from 50-100 thousand, and if the small business pays so much, it will go bankrupt. For small businesses, we have a special rate - 5,000 rubles. But this does not mean that at lower rates we work worse. All our tariffs use the same system, the quality of filtering is the same everywhere.
We are extremely apolitical. During the elections, our clients were Elephant, New Times, golos.org, Ekho Moskvy, Novaya Gazeta St. Petersburg, Forbes, Public Post, Vedomosti ... In general, we took all opposition under our wing. But even with great pleasure, we would work with the same CEC. But the CEC did not come to us.
The only criterion for us - the resource must necessarily comply with all laws. We basically do not communicate with sites containing pirated content, having a Nazi or pornographic orientation, with pharma affiliates and other Internet filth.
We thought that if we were able to create a system, the construction and operation of which would be cheaper than carrying out an attack capable of killing this system, then we would have eliminated the economic leverage of the attack. Attacking would be unprofitable. Based on this, we built the ideology of the development of our solution.
First published in the Hacker magazine dated 02/2012.
Publication on Issuu.com
Subscribe to "Hacker"
DDoS attacks appeared at the dawn of the Internet. My personal acquaintance with them began during the IT Territory project in 2003, when the game was just launched. She had a fairly aggressive advertising campaign, in response to which DDoS immediately flew in from competitors. Frankly, I was confused. Most of all due to the fact that the company that provided hosting services, not only could not, and did not want to fight the attack. Her representatives said that it was not their problem.
Literally overnight, we rethought the structure of the application and rebuilt it. The resource rose, but the attack stopped working at the application level. The attackers transferred the attack to the network level, as a result of which the entire network of the hosting company was disconnected. Then she came to us with a request to do something. The question “what to do?” Was followed by the answer: “Well, agree somehow, you know who is attacking you.” Naturally, we did not know and could not agree. And how to negotiate with the terrorists?
')
The company, which, it would seem, should take care of customers, did nothing. And not because she is bad and poor-quality hosting, but because she could not do anything.
The most popular DDoS attacks are, of course, attacks organized with the help of botnets. This is an affordable way to make the attack distributed.
The cost of an attack strongly depends on how it is implemented. The performer may be a student who wrote something himself and is ready to work for a beer, or maybe an organized group.
There are types of attacks that can cost, according to rumors, half a million rubles and more. We distinguish attacks of a basic type: up to five thousand bots, conducting at the application level, one strategy. For the performer there is nothing complicated - got WMZ, pressed the button, went to drink beer. It costs about $ 30–100 per day. But there are other kinds of attacks, when it is clear that the team is working, and it works 24/7 - for the result. If she fails to achieve a result, she constantly switches attack modes, changes her strategy, tries to find a vulnerable spot and break through. Of course, this is not far $ 100 per day.
Russia stands out among other countries by far more sophisticated attacks. Europeans are shocked by how complex our attacks are. For example, we have recently been approached by one company that operates on the Russian market, but all of its information structures are located in one of the leading European data centers. When the company approached us, the data center experienced serious problems and was unavailable. We prepared for the reception of the company on our network and warmed up the quarantine equipment, waiting for something extraordinary, because the data center was "dead"! Imagine our surprise when we saw that, although the attack was carried out at a level above average, it does not constitute anything special.
Legislation is extremely weak in matters of accountability for DDoS, which is why attackers feel that they are unpunished. For a qualified programmer, conducting DDoS attacks on an order becomes an absolutely safe profitable business, the income from which can exceed current salaries in the market tenfold.
Motives for DDoS are, as a rule, money and just personal dislike. De facto we live in an information society. The speed at which information is distributed affects it in a direct way. Blocking the source of information, you can irreversibly affect society. Accordingly, DDoS is an effective means of blocking any source of information for the required time.
An example is the site Slon.ru. The site worked, everything was loaded, but the attack did not fall. Such attacks are called combined. When they came to us, it was conducted at the network level, on the runway. When the attackers saw that filling the bar did not work at all, the application-level attack began. The botnet used for the attack included about 200–270 thousand bots.
The most subject to DDoS attacks are narrow enough areas of highly competitive Internet businesses. A good example is the pirated clones of Lineage II. Such services are a separate story, because they are purely commercial. If at two o'clock in the morning someone knocks to you in ICQ and, having allowed eight spelling errors in four sentences, requires (!) That you should help him immediately, you can be sure that this is him! Administrator Pirate Lineage!
DDoS-attacks are also those sites where it is difficult to expect them. We, for example, have such an internal meme - “cedar barrels”. An online store that, believe me, sells cedar barrels, has undergone a serious DDoS attack. This is a very narrow, highly competitive type of activity, which apparently brings a good profit.
The technical side of DDoS
Why do I need to classify attacks? To understand, sort through their work mechanism and take adequate countermeasures.
Our colleagues are trying to somehow classify the attack. On the same site you can find ICMP spoof, DNS amplification, and TCP SYN flood, TCP RST flood - the guys list the attack techniques. A lot of scary letters that for the average user do not carry any meaning. This classification does not suit us.
We classify attacks very simply: attacks on applications, attacks on a channel strip (speed is measured in gigabits / s), attacks on a network infrastructure (speed is measured in packets per second), attacks on a transport layer (TCP / IP stack).
The flesh itself is the application level. Why? Because the application level attacks have maximum leverage. The attack shoulder is the ratio of the resources needed on the side of the attackers to the resources needed on the application side (on the side of the defenders).
Take some average online store. You can find a link, a regular link, a certain number of hits per second to which this store will kill completely. There are many such applications, and in order to kill them, sometimes a botnet is not needed - a cell phone is enough not even with EDGE, but with GPRS. Four to five requests per second, and the application is thrown into the outflow and cannot exit from there until the server is restarted. This is due to the popularity of attacks at the application level.
There is also a transport layer - attacks aimed at the TCP / IP stack itself. This type includes attacks such as SYN-flood, RST-flood or FIN-way - a trendy attack now with incorrect connection closure, which, by the way, also exploits the vulnerability of the protocol specification, and not the implementation.
The most popular techniques include DNS amplification. It is enough to find any IDP based network service without handshake, send an N-size packet with a falsified source and receive an N x K packet in response. In this case, to implement a distributed attack, you need to have a list of IP addresses that have these services, one very well connected server on gigabit, which will give a set of packets to these "reflectors" with the shared IP address of the victim. The packet will go to the victim and reduce its throughput to zero and take it out of working condition. This is a DNS server - UDP 53, where you can do such a thing as a recursive query on a zone. By itself, it is small, but the answer to it will be long. To further increase the K-attack coefficient, it is enough to “feed” these servers with some large fake domain zones. Getting them recursively, with the fake address of the victim, you can increase K at times. The second option is NTP, time synchronization protocol, which also has similar vulnerabilities in poorly configured servers.
A SYN-flood attack appeared simultaneously with the TCP protocol. The first mention of her I met, in my opinion, in 1982. Oddly enough, it is effective to this day. The story develops in a spiral. One hundred and five hundred SYN requests per second is, of course, a stage long gone. Currently, with sufficient processing capacity, you can easily surpass the figure of 10 million packets per second.
At the time of sending a packet with a connection request, the partner must generate a sequence number (and this requires computation, since it must be a random, crypto-resistant number) and create a definite record on its stack. It requires resources, resources and resources again. Sending a packet by generating its traffic requires significantly less resources than actions performed on the server side. We get a shoulder attack.
The problem exists within the protocol itself, in its specification. When TCP / IP-stack was developed, no one thought that the Internet would grow to such a scale by the number of nodes, reach such speeds and, which is important, will pump so much money through itself.
There are attacks that do not use botnets. Distributed attack can be carried out using the mechanism of reflection. The classic of the genre is attacks of DNS amplification with reflection and power increase.
Attacks on the infrastructure affect everything that lies around the network infrastructure: routing protocols and the hardware itself, if the management modules have a public IP address.
What are attacks at the network level? Just fill the band at 56 Gbps - this is the clustery sort called. This is the last resort, when nothing helps. Such attacks are very expensive and extremely destructive, not only for the victim herself, but also for everyone who “stands nearby.” As a rule, they cannot last longer than two or three days, as they begin to cause problems even to the sources of attack - the networks from which it is produced.
Basic attacks made with the help of botnets, which have about 200 bots and do not know anything except “get root”, should not in principle be a problem for a competently written resource.
About DDoS Protection
You can protect yourself from any attack. We do not doubt it.
Usually, when a DDoS attack starts on a client, the hoster doesn’t find anything smarter than just turning it off, as he is afraid that his other clients will fall down too.
When a client comes to us, we explain that he needs to transfer the DNS to our IP address (which we allocate to him). Also, in order to avoid an attack on a direct IP client that is already lit up on the network, it is necessary to change it at least, and as a maximum - change this IP and additionally hide all IP addresses except ours using iptables settings or a firewall.
As soon as the DNS is rebuilt, filtering begins. And then the most interesting thing happens - filter training. Usually we set the bar for ourselves: after two hours, under attack, the client’s resource should start working. And in general, we maintain it.
The security system of our Qrator service is based on a variety of mathematical constructions. As usual, Yandex responds to the question “how is your search arranged?”? Yes, just! We take the text, tokens, dividers, build indies, we rank. We have about the same thing, only we solve the problem of analyzing and filtering traffic. Her decision employs a lot of people.
Behavioral analysis is one of the most effective methods of filtering traffic. We view the site as a transition tree. There are pages in the tree nodes, and in the edges we lay the transition probability and the transition delay. It is based on the simple fact that robots and people see web pages differently. When there is enough time for training, people “tread out” in these transitions certain seals — paths. All visitors who drop out of them, with varying probability, are robots. It seems to be all simple. On the other hand, if you estimate how much memory and computational resources will be needed for processing, you will understand that, probably, with current computing power, this is not so easy.
If a client comes to us already under attack, learning from attack ... is not impossible, but difficult. Often this requires monitoring by the engineer. That is why for the connection under attack, we are forced to take some additional charge.
We recommend connecting to our network before a DDoS attack. Of course, we try to minimize the training time under attack. With us, it is not a week, like Cisco Guard (this is our iron rival, which has been discontinued), but only a few hours.
Anyone who says that he has a false zero is a charlatan. False alarms (when legitimate visitors are blacklisted) are unfortunately inevitable. If only because there is a proxy, there are NATs, there are just people who do not behave like regular users. A classic example is site administrators. The administrator can load the server as 30, 40 and even 100 users.
We had one complaint about Cisco Guard: when you connect the attacked service to it, then regardless of whether there is an engineer there or not, the first day the service works in such a way that it would be better if it didn’t work at all. From this it became clear that it is impossible to protect against DDoS attacks at the application level without understanding the semantics of the application protocol. Semantic analysis is required, as well as behavioral.
We clearly understand that there is no “silver bullet”: what will work well in some situations will not work in others. The Qrator classifier is a complex set of algorithms that make up a voting system. We try to develop and add tools to the toolkit and, I hope, we will find some more effective methods in the near future. Some ideas are already there.
Approximately one million dollars will cost a piece of hardware from Arbor, capable of cleaning 10 Gbit. Plus a person, plus channel capacities ... At the same time, we see attacks at speeds above 10 Gbit / s about once a month and a half.
We tend to distinguish two types of bands: active and passive. In the active band, you can terminate and analyze any TCP connection and make a decision on it. The passive band is the band for which control you need to set a bit mask for which traffic will be cut. Thus, something intellectual can not be cut there. If we talk about the active band, almost all of our traffic providers block UDP from a specific address, if necessary, all ICMP or ICMP on a specific signature. On this strip, we quietly lived 57 Gbps. We are sure that we can live more. Such attacks do not cause any special problems, except for the need to pay for this band, that is, we are talking about a figure of more than 100 Gbps for the passive band. As follows from the situation with DDoS attacks on the Russian market, this is quite enough.
The advantage of Qrator (as a service) over purchased Arbor is that our solution is not a point solution. The network is built on BGP-AnyCast, we choose to install points exclusively trunk operators. We do not put points on public exchange simply because it does not guarantee the quality of service. The network is developing thanks to our own modeling algorithms. We build it so that it is possible to distribute the load on the network elements more or less evenly.
Inside the point of presence, the system is also scalable. The point is not one “piece of iron”, there are several of them. There is a quarantine equipment on which some attacks "land".
We created a model that allows you to mathematically calculate how traffic is distributed over the Internet when certain BGP announcements appear. This allows us to develop harmoniously and build a network that can really be balanced across nodes.
We are not tied to a single telecom operator and we try to distribute risks across all operators with whom we work.
We tried for a long time to deal with the TCP / IP stack, looked at Free BSD and Linux, and eventually we came to the conclusion that we didn’t like the stack in its current state. We have our own lightweight version of TCP / IP, which behaves very well on current short-lived protocols, fast TCP connections.
We do not hide that the filtering node is running Linux. Linkus is a container in which the platform is managed and the mathematical transformations necessary for behavioral analysis are performed. A substantial part of the TCP stack lives in the TCP card itself, which is why, in fact, we have such good rates of speed / packet handling. One of our filtering sites is able to shovel 6 Gbps of traffic.
You can defend against basic attacks. To do this, you must have a dedicated hosting, as well as the ability to compile the modules and your version of the web server. There are a lot of articles on this topic, and I will probably refer you to my 2008 article (you can find it on the blog at highloadlab.ru). This is one of the first articles in which it is easy to explain what and how to do. I also recommend that you familiarize yourself with the presentation “A Practical Guide to Survival in DDoS”, which we showed at Highload ++ in 2009.
We tried to write articles on Habré and tell at an industry conference how to defend oneself against basic attacks. But, unfortunately, this had no effect.
About botnets
DDoS is one of the ways to monetize a botnet, but far from the most profitable. There is also spam, fraud, advertising clips and so on.
I listed the botnet properties on the slide for one fairly old presentation. When I did that slide, it seemed to me absolutely correct:
- Greed. A botnet tries to inflict as much damage on an application as possible per unit of time.
- Damage Botnet is not a browser. This is a kind of HTTP stack built into the worm. As a rule, he does not know how to put correct headers, does not possess a JS-engine or possesses in a limited form.
- Self-preservation. A botnet is a valuable resource, and any actions that lead to a reduction in its size bring direct financial losses to the attackers. A botnet tries not to produce actions that could be unmasked and reflect on the parent system.
- Transnationality Botnets scattered around the "ball."
- Extremity
A couple of years ago, we first saw a slow botnet that was not greedy ... He made one absolutely legitimate request every five minutes. We were surprised, but at the same time the botnet, which counted 75 thousand bots, still caused problems. Try filtering THIS.
Now, of all the points listed above, only one remains - the botnet’s desire to self-preserve. Botnets are no longer greedy, not stupid and not flawed. Now we are dealing with full-fledged minimized web browsers with Java scripts, redirects, cookies.
Distributing commands to members of the 20,000th botnet, given that the bot itself is the initiator of the connections, is not a trivial task. As a rule, control panels, to our surprise, are written on the same LAMP Stack (Linux, Apache HTTP Server, MySQL and PHP). Until 2010, the deployment of the five-thousandth botnet in the direction of the resource of the victim took 30-40 minutes.
In 2010, botnet management began to be organized using P2P. The guys began to simply distribute commands to the super-fast: within five to six minutes a botnet, numbering 10-20 thousand bots, can extend the team inside and turn around for a resource.
Botnets try to imitate user behavior as precisely as possible in order to make it more difficult for them to be detected and filtered, to isolate the botnet body and block it.
For example, not so long ago there was a surge in the activity of the MinerBot botnet, which produces BitCoin. He comes to cover pages without a referrer, randomly, follows links and really creates problems for solutions like Cisco and Arbor. They are not able to filter out MinerBot because it does not have any of the flaws that these solutions are aimed at detecting.
Botnets also ceased to be transnational - downloads are easily sold by region. The first time we saw this was in 2009, when a botnet for 1500 heads “came to us”, and everything was clean in the CIS.
"Falling asleep botnet" - so we call a rather fashionable attack. The botnet detects that it is all filtered, distributes the command and stops the attack centrally. After that, the random member of the botnet sends test requests waiting for the filter to turn off. As soon as it turns off, the attack resumes within three to five minutes. This is dangerous because such an attack can last forever - no resources, from the point of view of the bot, it does not consume.
Different botnets are very different from each other. The attack technique itself is constantly changing.
With millionaire botnets, there is a very interesting situation. In the past few years, the number of those who would like to acquire their botnet has increased significantly. A simple experiment: put Windows XP SP1 on an honest IP address. How long will he live before something lands on him, even if he doesn’t open the web browser? Maximum five minutes. There are many teams that are struggling to increase the body of a botnet, and the supply of vulnerable systems is extremely limited. Accordingly, the number of botnets is growing, but their size is slowly but surely decreasing. Botnets are already beginning to intersect, that is, one computer is a member of several botnets at once.
There are fewer and fewer botnets on tens of millions of computers. They have quite a Jedi. :)
We do not have the ability to do reverse-engineering of the botnet code, because we do not have the administrative capacity to seize his body and, most importantly, we do not have our own specialists capable of doing reverse engineering of the code oriented to Windows systems.
About Highload Lab
The idea of ​​practicing DDoS attacks originated in Moscow State University. We looked at how things are going with the resilience of government resources and the resilience of web applications in Russia as a whole. It became clear that our services are likely to be in demand. After all, a fallen online store is a problem only for its owner, but a fallen tax inspection is a problem for the whole country.
Start research - it was my personal initiative. The university provided the infrastructure, I bought the equipment with my own money.
In 2008, we had an idea. In 2009, a beta version of the product appeared, which we tested in open beta mode for most of 2010. We hosted any project in distress on our site. It became clear that we cope well with this task, even with a limited university infrastructure. We, for example, helped the Vedomosti newspaper. Was cool. :)
The need for commercialization pushed us: in June 2010, when the maximum capacity of the university network was 10 Gbps, an attack of 12.5 Gbps fell on us. The attack showed that the filters are coping, and we can easily overcome a more powerful attack, but channel capacities are needed. This is a valuable and expensive resource, but I also do not want to lose ... We had some of our accumulated funds, for which the channel capacities were purchased. Also, additional equipment was purchased.
We were lucky with the launch - we had a wonderful stress test. That is, on September 1, according to the plan, I just put the last entry point, and on September 2, Habrahabr came to us under the attack of 6 Gbps. We got a free stress test.
Traffic is one of our main expenses.It is spent not just a lot, but a lot.
The company works in several directions: we develop custom-made high-load web applications and advise on their creation. The second direction, the most promising for us, the most dynamically developing, is our “boxed” product, the Qrator traffic filtering system. In it we invest almost everything that we earn.
At the moment, our company employs 12 people. The non-technical staff includes eight engineers and four other employees. Two of them are freelancers from Moscow. At the beginning of the year, if everything goes well, we want to invite two more engineers to the company. Just like Yandex, we are looking for mathematicians who can program, work with data (structured and poorly structured).
Unfortunately, we are not engaged in reverse engineering, but we see that each attack has its own signature and logic.
We have been in existence for a year and a half. It was not an easy time. At some moments it was very difficult both financially and morally. But during this time we figured out what issues arise during the operation of the service, understood how to create a tariff stack. Since the service is new, no one knows how to sell it. All offers on the market have certain flaws.
Highload Lab is profitable. This year we made major technical changes - we developed a new version of our specialized network processors - and actively developed partnerships with all interested companies: hosting companies, telecoms.
One of our goals is to provide protection for small businesses. This is the most unprotected layer of attacks. Many companies take for protection from DDoS attacks from 50-100 thousand, and if the small business pays so much, it will go bankrupt. For small businesses, we have a special rate - 5,000 rubles. But this does not mean that at lower rates we work worse. All our tariffs use the same system, the quality of filtering is the same everywhere.
We are extremely apolitical. During the elections, our clients were Elephant, New Times, golos.org, Ekho Moskvy, Novaya Gazeta St. Petersburg, Forbes, Public Post, Vedomosti ... In general, we took all opposition under our wing. But even with great pleasure, we would work with the same CEC. But the CEC did not come to us.
The only criterion for us - the resource must necessarily comply with all laws. We basically do not communicate with sites containing pirated content, having a Nazi or pornographic orientation, with pharma affiliates and other Internet filth.
We thought that if we were able to create a system, the construction and operation of which would be cheaper than carrying out an attack capable of killing this system, then we would have eliminated the economic leverage of the attack. Attacking would be unprofitable. Based on this, we built the ideology of the development of our solution.
First published in the Hacker magazine dated 02/2012.
Publication on Issuu.com
Subscribe to "Hacker"
- 1 999 . for 12 numbers of paper version
- 1249r. for an annual subscription to iOS / iPad (Android'a release soon!)
- "Hacker" on Android
Source: https://habr.com/ru/post/158945/
All Articles