📜 ⬆️ ⬇️

Conference DEFCON 22. "Mass scanning of the Internet through open ports." Robert Graham, Paul McMillan, Dan Tantler

My name is Rob Graham, I am the head of Errata Security, a company that deals with Internet consulting. Today we will talk about how to scan the entire Internet and why it is needed. Until today, there were few tools to solve this problem, so we created our own tools. The Internet is small enough - there are only about 4 billion addresses.

Scanning the Internet is quite simple - you sit down in front of your computer, launch the console with the command line and enter the subnet address. And you watch how your screen is filled with data, and the lines all run and run on. As a result, you get a list of open ports of devices with different IP addresses.


')
Why do I need to scan the Internet in the context of protection? If you are concerned about security issues, this should be done to get an answer to the following questions:


The existing tools for finding the vulnerability of specific networks and equipment are rather slow, but mass scanning allows you to get the characteristics of the vulnerability of over 100,000 devices fairly quickly. An important problem that needs to be solved is the identification of equipment that is used to communicate with NTP servers during DDOS attacks. A lot of home equipment is vulnerable due to the fact that D-link routers are not strong enough protection. Just look at the D-link network to see how many botnet systems exploit their vulnerability. Scanning SSL certificates is also useful because it reveals outdated certificates that are prone to errors and vulnerabilities. So scanning everything you can “reach out” is an important task.

Internet scanning is also needed in the context of prevention. It helps to reveal Deepnet - many Internet pages that are not visible to search engines. These pages are generated by user requests and may carry malicious information.



Try scanning random ports by running the "- banners" bulk scan command, and within a few minutes you will find something that can be hacked easily.

In fact, scanning the Internet is useful because:


What you need to know in order to scan the Internet? First, you need to know the theoretical part of the physical infrastructure:


This means that we are overpaying the provider because it charges us for guaranteed bandwidth, that is, for guaranteed, not actual data. This is due to excessive packet size. If we transmit 22 or 33 bytes, they are still packaged in a 40 or 44 byte packet. The user almost never can reach the full transfer capacity given in 1 gigabit, because in reality it transfers no more than 524 megabits per second. But due to fixed packet overflow, the data is oversized and this stock is not used at all. But we pay for it. Even if we have a perfectly tuned switch, we still will not be able to use the full bandwidth of the network, and I do not know why this is happening. Confusion exists in the system of payment of bills for Internet services.

How are traffic bills generated by Internet providers?





For example, in Germany there is a CCC club, which provides users with a speed of 100 Gbit / s. I could not test this network, but maybe this year I will take my 10 Gigabit Ethernet card with me and check if this is true. But the problem is that when we send too small packets, we thereby violate the existing agreements between the peers of the same network.

Consider the physical network infrastructure further.

Private VPNs are able to adapt to the load of small packets. Ethernet struggles with small packets, and speeds above 500 Kbps are often difficult. If your switch is able to work at such a speed, this does not mean that the rest of the infrastructure can support it. In this case, it can be helpful to disable the flow control by the Flow Control, in which the transmitter brakes the data transfer if the receiver is not ready to accept it.

In some cases, packets can be lost - transmission at a speed of 500 Kbps does not guarantee that all packets will reach the Internet. Scanning allows you to identify ports that use packet loss. You can use only those ports that provide the same reception and transmission: if you send 10 thousand packets, you will also receive 10 thousand packets. Therefore, I mainly use speeds of up to 150 Kbps, and sometimes even 15 Kbps, this allows not to think about the integrity of the packets.

A big problem is complaints of abuse, Abuse Complaints. This term means that someone has marked you as a source of spam or other malicious activity. Often this happens with companies when the addressee does not want to receive more letters from you, but cannot unsubscribe from the mailing list, because your company did not give him a link for this. It marks your mail as spam, and it harms the overall reputation. This can happen when scanning a network. You can get Abuse Complaints, and your ISP will get upset. Or you break the agreement between the feasts, you will not be allowed to play the role of a feast. However, there are much worse things:


The existing network monitoring methodology tracks incoming traffic. If you use a scan, your incoming traffic will be large and you will be suspicious. It is believed that hackers can be traced this way, although it’s like looking for keys lost in the bushes under a street lamp just because it's lighter there.

What should Internet service providers take seriously? To the fact that some networks use blackholing (“routing to nowhere” when such routing packets are deleted due to “No route to host”) for the whole autonomous AS network.

The exclusion list is required when scanning, since we do not want to scan other people's mailboxes and private network segments. Creating an exclusion list is done using the scan parameters in the command line:

/etc/masscan/masscan.conf exclude = 224.0.0.0-255.255.255.255 exclude-file – exclude.ips 



An important thing is to create a public list of exceptions. We would very much like to create a public list of security experts, but most of those who send us a request to participate in the program are usually asked to remove them from this list. They are afraid that someone will know their IP or corporate network addresses and try to hack them. Fortunately, BGP networks have all this information in public access, which is laid out in a fairly elegant format and is accessible to everyone. People need to understand that scanning the Internet will only benefit them and in no way affect personal information that they do not want to show anyone. Unfortunately, most have to prove this, because they confuse scanning and hacking. And in general, people find it difficult to believe that you can scan the entire Internet.

For example, a company has a certain network, which we scan at their request, but they also have a subnet in which important information is stored. So, after seeing how the scan goes, they get scared and say: “You even scan hidden networks, you see a range of ports and addresses, so you can hack us”!

An interesting story happened six months ago. I scanned the network for one customer, and he was woken at night by a call about an emergency conference called due to the hacking of the network. He called me and I had to calm him down and explain that the scan had nothing to do with the hacker attack, they just found their vulnerabilities before we. Often, clients believe that as soon as they give us permission to scan, some security gaps immediately open up, and hackers immediately climb into it.

Another case was with a guy from Australia. He noticed that when we scanned the network, we sent him a request for a connection in the form of a single SYN packet, and called me, they say, who we are and on what basis we are doing this. I explained everything, told him the address of our site, where there are all the rules and regulations, said that we do it absolutely legally, according to customer orders. He did not want to listen to anything and began to threaten us with the Internet police, that he would immediately call to where he should be and we would all be arrested. It’s just some crazy person who didn’t understand that if we were engaged in scanning illegally, in an hour we would have been caught by everyone, because we are acting completely openly.



Such complainants are often just stupid. They do not understand that the vast majority of processes taking place on the network, all ports, routers, switches, sessions are constantly open and are not protected by any encryption. Otherwise, the Internet could not work at all if permission was required for every action. If a person is afraid that his bank card data may be stolen, it is better not to use the Internet at all. And this is happening against the background of the fact that people are not able to simply configure their devices so as to close the existing gaps. They leave them open to everyone, and then they are surprised that they have become the prey of hackers. I want to show you the letter we received with such content: “The infrastructure of the Woori financial group is classified as“ class A national security equipment ”and unauthorized access to this equipment is prohibited by relevant laws and regulations”. This company is located in Korea, and we first learned about it from this letter, because they not only sent a complaint to us, but also explained their actions by letter. I first met an entire organization that wants to have access to the Internet while securing all its equipment. Why then go on the Internet at all if it cannot be done with closed ports?

An important aspect for our work is close cooperation with Internet providers. We have to be friends with them, otherwise we will not get an effective scan. We offer them free advice on Internet security, they help us correct the list of complaints received. That is, the provider understands who and why has complained about us, and rejects unfounded accusations against us. Together with them, we are creating a SWIP project - “Who is who on the Internet” with a list of verified IP addresses, and we put in our “black list” those who insist that we be banned for scanning.



As an alternative, helping to avoid some misunderstanding, you can create an anonymous virtual dedicated VPS server. It has the following advantages:


What does masscan technology look like?

It is similar to the nmap utility, which is designed to scan IP networks with any number of objects and determine the status of ports and their corresponding services:


But the masscan is not like nmap:


Nmap is the best scanner - its NSE scripting engine is very flexible, and scanning multiple hosts without problems. Masscan is designed for large networks, as this program is much faster and better scaled.

Masscan has its own TCP / IP stack:


This is how hacker attacks are performed with address spoofing, the so-called spoofing attacks. Suppose we have a host A - an attacker, a host V that is being attacked, and a host O, whose IP address the hackers want to use for the attack.

Host A sends a SYN packet to host V, but the return address does not indicate its IP address, but the host address O. The attacked host V responds to host O with a SYN / ACK packet. But Host O did not send anything to Host A and therefore must break the connection with the RST packet. Suppose that host O did not send such a packet because it was overloaded, or turned off, or is protected by a firewall that blocks SYN / ACK packets.

If Host O has not sent the RST packet and has not interrupted the attack that has begun, Hacker Host A can interact with Host V, impersonating Host O. Therefore, any authorizer, captcha checking and so on becomes useless if the user's firewall is not configured correctly.

Thus, RST packets protect IP connections from establishing communications, that is, they respond to packets with a SYN-failure. With their help, we can protect various IP addresses from spoofing, or install a protective filter for a specific range of ports.

Now let's talk about the teams that run masscan.

Scanning multiple devices is done in such a way that security protocols are not violated:


Sometimes, to avoid problems, manual TCP / IP connection configuration is performed:


This is what the banner verification team does:


At the moment I am using something similar to NSE scripting, but I will soon switch to C-based programming.

You can also use load testing. This can “penetrate” the protection of firewalls and is therefore important for testing their ability to provide security. In this case, the commands - - infinite, - - banners, - - sourse-ip <range> are useful for quickly scanning a large number of devices.

Usually nobody uses it, but in our program there is the possibility of using outgoing binary files, for this the command is used:

 – oB foo.scan  –oX foo.xml 

Then the conversion is performed:

 masscan–readscan foo.scan –oX foo.xml 

This method provides a more compact scan. In addition, if there are errors in the outgoing data, it is easier to correct them in binary format.

Another useful feature is spoofing scanning. IP spoofing consists in spoofing an IP address in the packet body so that the response packet is intercepted by the hacker address. This technology is used by hackers to intercept traffic between hosts on Ethernet networks.

Spoofing scan consists of the following:


Here's what the scan results look like. On the first picture you see the window of our program, on the second - the result of its work.

The result of testing for the threat of Heartbleed shows that as of April 10, the vulnerability was found in 600,000 systems, and in July 300,000 systems were still vulnerable, most of which were hardware devices, that is, the computers themselves, routers, webcams and servers. That is, you will not see their vulnerability, if you check them with DNS names, only scanning by IP addresses helps. We also scanned mainframes — large fault-tolerant servers, such as TN3270 Telnet –over-SSL over port 992. You can take a look at @ mainframed767 and see interesting things like the authorization window of the IBM main server user.

The third picture shows the results of scanning banners. Now I will try to show you our program in action. To do this, I open the main window and use the command line to set the address of the server I want to scan. In some cases, the server does not respond.







Now Paul will try to log in and demonstrate the scanning capabilities.



Paul says that if you have any questions about using the program, you can contact him directly and get the necessary explanations. As an example, Paul crawls the Internet through a VNS 5900 server, which takes 15 to 20 minutes.
The advantage of our program is the ability to obtain a list of vulnerabilities without the need for authorization on the network or on each network device. We test the system from the outside, not from the inside. Using scaling allows you to check huge amounts of Internet networks, including clouds, and it costs less than 16 cents per hour.

Right now I am asking for a slow defcon network scan on port 80 at 10 packets per second, and the result immediately appears on the screen.



At the moment, we see how many open unprotected ports in the network are devices with corresponding IP addresses. And a hacker can use these IP addresses for his spoofing attack.



This procedure does not interfere with the network, which is subjected to scanning, the user can run any application. So, defcon network scanning took a little more than a minute, and we identified all existing vulnerabilities by simply scanning port 80. By setting the packet size, you can speed up or slow down the scanning. We told you everything you need to know about the masscan program, and if you have any questions, email us or twitter @erratarob and paulm .


Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to friends, 30% discount for Habr users on a unique analogue of the entry-level servers that we invented for you: The Truth About VPS (KVM) E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps $ 20 or how to share the server? (Options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper? Only we have 2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV from $ 249 in the Netherlands and the USA! Read about How to build an infrastructure building. class c using servers Dell R730xd E5-2650 v4 worth 9000 euros for a penny?

Source: https://habr.com/ru/post/351928/


All Articles