Conference DEFCON 16. "Cisco iOS Development Crime." Felix Lindner, head of Recurity Labs
The next item in my talk is about a frequently asked question about how Cisco centers work. Why exactly Cisco? Because they occupy 92% of the sale of routers market cost from $ 1500 and 72% of the switches market. We will also discuss Juniper software, which, in terms of both hacking and protection, is open source software based on FreeBSD. Therefore, for us it is not interesting. If we look at cheap home routers from the same points of view, then we will see that they have classic Linux embedded.
Today we will examine in order:
Cisco IP routers and iOS infrastructure;
Cisco internal operating system
existing approaches to the problem of hacking and security of the company's products;
New analytical approach: proposals, opportunities, changes;
public offer;
future work in this area.
The Internet and corporate networks typically use IPv4 and IPv6 network protocols — the former is more common, the latter is implemented too slowly. The design of IP networks uses intelligent nodes to provide routing. This protocol configuration cannot be changed. ')
“Flat”, peer-to-peer networks operate according to their own rules.
Why is network forensics necessary? Its main goal is to study the current state of computers and digital storage of information. Like any kind of forensic science, it consists of obtaining evidence, extracting information from them and analyzing it. For existing computer operating systems, tools and methods for conducting investigations have been developed, and nothing similar exists for networks. Therefore, we cannot say for what reason Cisco's “box” failed, until we investigate all the circumstances.
Who needs to hack routers? If you hack a regular computer, you will get access to it. Having cracked the main computer, get access to the working group of computers. If you hacked the switch, you will get access to all computers connected to it. If you manage to hack a router, you will get access to everything that is on the network.
As soon as one of the computers on the network is under threat of hacking, the switch blocks it and puts it into quarantine, and the hacker cannot do anything.
If you manage to hack a switch, you gain control over a group of computers connected to it, and the rest of the network is not able to determine who took control of the switch. Therefore, you will be granted access to the rest of the network, as if you are an authorized user with the right to do so.
If you have taken control of the router, the whole network is in your hands. In this case, a pleasant bonus will be unlimited access to the Internet. The Internet itself can be represented as a huge number of routers.
Below is a Cisco-style cooperative network and network security diagram. It depicts a variety of autonomous systems, united around a single network core. Here are all sorts of business buildings, government and private homes.
And in the upper right, a group is shown that is marked with a symbol (_x_). If someone does not know, this is the international designation of the word "ass." It is from there that I will try to hack our network. Firewalls, IDS, IPS are located on the borders of these autonomous systems - they are not needed by everyone, but all use them. If we look at the core of the network, we will see routers connected to autonomous systems and providing data routing. They have input and output filters, authenticators of network users, etc., which protects the kernel from third-party penetration. Four routers located in the middle of the kernel trust each other. They ensure the interaction of all network participants and simply have to trust each other, otherwise they will not be able to work. This suggests that the network is a strict hierarchical structure, and its security is also subject to hierarchy.
An element of the system can be protected from devices subordinate to it, but it is very difficult to resist upstream devices. You can accept the message, ignore it, or lose the Internet altogether by entering into an argument with the parent. The ability to defend against an element with the same rights is generally very rare, since the lack of trust hinders the work. By capturing control over the main link of the hierarchy, the core of the network, you can control all subordinate structures.
If you can control one of the routers, which is an equivalent kernel link, then the system will become yours. To counteract this, there are security protocols. They ensure that no one can modify and replicate protocol messages or forge equivalent elements of the system. But if someone still succeeds in doing this, the security protocol cannot do anything with it. You will know that you have been hacked, the protocol will make it clear, but you cannot do anything about it. Therefore, the user has to choose between network availability and security. For example, you can get off the couch and go to the bank to make a payment manually, or stay on the couch and do it over the Internet, knowing that your money can be stolen during their journey through the network.
For companies, this situation is much more dangerous, because they can not perform any actions outside the network. And if you disable security protocol checking, the network simply stops working.
However, the security protocol provides the hacker with the ability to track the channels through which it is distributed. If the user can control the path through which he is connected, this can be called a “source of routing”, and he is sure that no one else on the Internet uses this route, then the user will get the necessary power over the network.
In IP networks, network nodes that have “intelligence” make decisions about sending information, and the border nodes where users are located cannot control the flow of traffic. If you hack the network node, you will gain control of the network. That's why we need forensics networks - we find an opportunity for hacking and ways to resist it.
There are 3 types of network attacks:
protocol based attack;
functional attack;
the introduction of malicious codes.
Protocol-based attacks represent the injection of control protocol messages into a network (attack of routing protocols), when an attacker becomes part of the internal structure of the network and affects the forwarding of messages. An example of this type of attack is the corruption of ARP, DNS, internal OSPF, EIGRP, and hacking of external BGP routers.
Functional attacks are distorted network functions. Network configuration problems include false passwords and false SNMP network management protocols, as well as access codes posted on public forums. People often receive messages asking them to change their password to ensure security, but they do not always pay attention to the sender of such a message. A person changes the password at the request of the hacker, and he gets access to the network on his behalf.
This is followed by hacking of the access system check and redirection of vulnerabilities, which allow Cisco HTTP protocols from version 16 to perform on themselves. Hacking a vulnerability check can be performed using the SNMP v.3 HMAC information integrity check protocols or memsnp - myHMAC, PackHMAC, PackHMAC_len. Such hacking makes the system think that the package contains verified, and not malicious information. In addition, the network can be influenced through hidden vulnerabilities. SNMP is the most popular standard for routers, and people use nothing else.
If the normal password was used in SNMP v.1 / 2, then version 3 uses encryption. If you manage to implement your own encryption there, everything is in order. If not, you will not be able to find the password. SNMP v.3 HMAC has been used only since 2008.
HMAC is a message authentication code using one-way hash functions. In cryptography, it checks the integrity of the information in order to prove that your password is yours, and not invented and inserted by a hacker into your web authentication form. That is, the data stored in an unreliable environment, prone to attacks, were not modified by the attacker. If the hash algorithm is simple, it can be hacked, if it uses an algorithm with a greater length of the cipher, for example, SHA256, you have to work. However, most routers still use simple encryption, which is why cryptographers need to regularly update the operating system to complicate it.
People also use Debian SSH packet keys — a network protocol that allows you to control a router or computer remotely through a command shell. It completely encrypts the entire flow of information, including passwords. Another Cisco vulnerability is the order of information exchange. Data packets sent by devices are lining up in front of the router, and if someone tries to snatch a packet from the queue, the entire transfer will stop.
Consider the use of binary codes. Vulnerabilities of routing services include:
using the TFTP exploit (simple data transfer protocol) of the Phenoelite hacker group;
use of the HTTP (network protocol) exploit Phenoelite;
using the Andy Davis FTP exploit.
Routing protocol vulnerabilities include:
use of the exploit OSPF (Dynamic Routing Protocol) Phenoelite;
use of exploit IPv6 (Internet Protocol) by Michael Lynn.
Thus, you can attack as a routing service, and protocols. Improving protection leads to improved exploits, so this struggle is endless.
Consider how threats are detected and monitored. This can be done using the following functions.
For example, in the protocols for checking the integrity of SNMP information, there is a voting mechanism that activates pop-up notifications (traps) on the screen. These protocols control the routers and report what happens to the device during operation.
Syslog also sends pop-up free-form notifications, and if you pay attention to it, you will notice if someone tries to change the configuration of your router.
Configuration configuration messages can also be traced; there is real-time monitoring of routing paths that allows you to see changes in the data transfer path. The final tool is to calculate the amount of traffic. It is not designed to monitor security, but serves to prevent the loss of traffic for which the customer paid. However, counting the amount of traffic can indicate who in the network uses the largest amount of data, and possibly produces a network attack in this way. Usually, providers block a user if too much traffic is coming from his computer.
The following table shows who can detect which threats.
Malicious attacks can be detected using SNMP, Syslog, monitoring routing and traffic counting. Internal attacks on the router can detect SNMP, at least - Syslog, monitor the routing and traffic counting.
External attacks are also detected using SNMP, Syslog, route monitoring and traffic counting.
Unauthorized entry into the configuration settings of the router can be detected using SNMP, Syslog, and the device manager, which displays configuration messages. The threat of access checking can be detected using Syslog and configuration messages. The implementation of malicious binary codes can only be detected by monitoring configuration messages, and if these codes are fairly simple and affect the configuration of the router itself. Thus, the most dangerous are attacks using binary codes. What do these codes do? They distort the execution of such procedures as hidden identification of client access, the mechanism of network entry (registration in the network) and violate the functionality of the firewall. Codes can change the data structure: change the level of access to the Cisco VTY virtual interface, add a false virtual interface (Michael Lynn's attack) and suspend data transfer processes. For example, if you do not need any of the processes occurring in the router, you stop them.
However, this can only be detected after the attack has already taken place. What else can binary codes do?
change the current configuration and configuration of the state of the machine, for example, the network management protocol SNMP;
Download TCL hidden control commands (the latest iOS versions support scripting in this language);
intercept control of TCP ports using these commands, and in some iOS versions TCL commands even allow you to “freeze” processes in a virtual VTY interface.
Usually, such procedures are resorted to by people who have been denied the right of access to the highest level of the system and are trying to return it. Simply put, a person runs a malicious script that returns former privileges to it.
What is needed to investigate the use of a binary code? Find evidence of interference with the system, recover information from the raw data and analyze this information. To do this, you need to be well versed in the features of Cisco's iOS.
What memory is used in devices of Cisco operating system, for example, in 5000 series routers? The operating system boots from ROMMON (ROM Monitor). In this mode, the router is suspended.
The operating system image is loaded from a flash drive or network into RAM. This image can be self-extracting and contain firmware for the additional equipment of the router, that is, the modular hardware - network cards, firewalls, supervisors.
The configuration is downloaded as ASCII text from non-volatile NVRAM or from the network. It is in a “disassembled” state and is mixed with the version image depending on the default configuration settings. For example, for two different routers, the default configuration may also be different.
All this is contained in RAM. Configuration changes cause an immediate effect, and the new configuration is written back to NVRAM using the command. Thus, if you are going to hack a router, you need to do this at boot time. Configuration changes are not automatically saved anywhere, so experts may have a question like: “what have we changed in this configuration in the last two years”?
Consider evidence of intervention in the system. In a normal OS, most of the evidence is not recorded anywhere, but preserving the image of the hard disk allows it. However, users practically do not resort to such a procedure, since they do not feel the need for it. Therefore, there is a very weak ability to track changed data.
In Cisco, almost all the evidence and events are recorded, they have such a protection system. All that is needed to obtain evidence is a memory image obtained upon request or at the time of loading the operating system. The image recorded in case of a router breakdown or in the event of malfunctions in the work allows us to understand the reason that caused them. Therefore, in case of errors, the system reboot is used as the default procedure. That is, if your attack was detected, the router will try to reboot as quickly as possible in order to restore the initial configuration settings.
However, the reboot destroys all evidence of what caused the problem in the system. So, in the Cisco system, non-fixable evidence of intervention is also possible. This happens in the flash file system if a hacker statically modifies an iOS image, in nonvolatile NVRAM memory, if a hacker changes the configuration and writes it back to NVRAM, and in both these areas, if the hacker uses binary codes.
Hackers do not use this approach because they can leave traces of their intervention in the modified system image and in the modified boot configuration, but resort to manipulating RAM, in which there are no traces of intervention.
A good thing is to use debugging functions to capture evidence. iOS can record full core memory dumps, that is, at certain points, take snapshots of information about the state of the system. Dump targets: TFTP, FTP, RSP, Flash issues. The complete memory dump includes a snapshot of the main memory, a snapshot of the IO memory and memory of the PCI slots. Memory dumps can be serious evidence for computer forensics. This technology is widely used by Cisco engineers to protect their equipment.
Kernel memory dump is pre-configured in the system settings. Configuration changes do not affect the functionality or performance of the router. All iOS of different devices are configured to work with a core memory dump located on one or several central FTP servers.
Such an approach reduces the requirements for monitoring the system, retains evidence of interventions in the operation of the system and allows you to track the relationship of problems between different routers. You can write to the core memory dump everything that happens in the OS of the connected routers. This saves you from having to jump out of bed at night, when one of the routers suddenly reboots or a part of the network ceases to function. You can find out what happened by coming to work in the morning and analyzing the system core memory dump.
Why has this technology not been applied before? Because nobody used core memory dumps, except Cisco engineers and developers of malicious exploits.
Analysis of the kernel memory information is that the dump data must be translated into a clear form using a dump reader. This allows you to get answers to questions:
What happened to the router when the memory snapshot was taken?
What processes managed the data at this moment?
where did this data come from?
What data packet caused the router to malfunction?
The following requirements apply to the memory dump analyzer program:
it should be 100% independent — no Cisco codes, no attached databases for analysis. Because if you investigate damaged Cisco codes, you should not do this using the Cisco codes themselves;
must be unbiased (no abstract assumptions about the cause of the events) and have the ability to copy large amounts of corrupted data;
not to be infected itself, for which purpose such a program should not be written in C;
if it has the properties listed above, it should be used with such a useful tool as CIR - Cisco Incident Report, or Cisco event log.
When analyzing the kernel outside of the Cisco iOS system, you need to keep in mind that you have to work with one large binary binary ELF file, which is an image of the system. Essentially, it is a large, UNIX-based program that is loaded using ROMMON, a BIOS analog. It runs directly in the main processor of the router, and does not allow you to adjust the priority of execution, as it happens in a regular computer with a multi-core processor. Virtual memory card is used to a minimum.
The processes are more like a continuous stream, so the virtual memory card will not help much. They continue to completion, unless they are interrupted by a critical event. This is a global dataset spreading throughout the system that cannot be dispersed. Therefore, we can work with one "bunch" of information, which is very convenient.
The iOS image as an ELF file contains all the necessary information about the memory map of the router. The image serves as a kind of drawing on which the core files are located. Using an intact image guarantees viewing codes and information that are read-only. Thus, we can find patched, rewritten data on this map, not belonging to the “native” system, because the forbidden changes were made to them. We simply compare the image of the system with a memory dump and see the changed data.
The program we developed by Recurity Labs SIR successfully counteracts the Topo's DIK hacker program, since it analyzes the mismatch of virtual address codes in the segments of the ELF file and the memory dump. In this spyware program, they make up 4 elements in each broken code.
Next, we perform a reconstruction of the entire "heap" of information that the core of the system uses. This information contains simple metadata for debugging processes, 40 bytes in one block in iOS versions up to 12.3 and 48 bytes in iOS version 12.4.
Reconstruction of the entire array extends the period of the event in real time, which allows us to notice the slightest deviations that could remain in the shadow of our attention. Our program places the verified data on a separate sheet and checks the following. And if there are differences in the data addresses, this affects the block size. There is also a heuristic analysis of rarely used areas of the array, which may also contain malicious code. As a result, we find blocks, during the processing of which, an increase in memory usage was observed, which in turn indicates their change. And we can determine which processes “eat up” memory, and stop them.
We also have a list of processes extracted from the general list of iOS processes. It allows you to determine the location of the stacks in which they occurred. We identify the process stack and the address from which the command to change it was received, we obtain the history of the use of the processor to perform this process and a description of the event that was processed by the processor. We compare the stack, the memory and the process and find out what was wrong.
Almost all of the described analysis technologies can be applied if it is possible to reconstruct the two above-mentioned data structures.
We can extract any TCL script from a memory dump, for which the TCL decompiler is used, which turns, translates the program process into its source code.
We also have random applications that may come in handy in computer forensics. They allow you to fix processor malfunctions, determine the fragmentation of blocks, find out what processes are happening and where they occur, and can find the attacking processes. Also useful are tools for system research, for example, point correlation or analysis of system memory.
Consider a memory for forwarding IOS packets. The operating system allows you to use a router as a process switch, a fast switch, an element of a system process, or a hardware accelerator switch. All these functions, except the hardware switch, use IO memory. It is recorded in a separate memory dump. By default, about 6% of the router's memory refers to virtual IO memory. Increasing the size of virtual memory can speed up the data transfer process. Switch hardware uses PCI memory. It is also written to a separate core dump.
The so-called circular buffers of the memory of the router are grouped by size - small, medium, large and huge. For example, TCP packets use large memory buffers, while exploits use small ones. And these little packages remain in memory for a long time. Interface devices have their own memory buffers for local traffic control. IOS tries not to add packets to memory, because new traffic does not automatically erase traces of old ones.
To free up memory, we extract traffic from virtual memory into a PCAP file. This is the file that contains the packet data of the network. This process is performed using CIR packet dumps. In this case, the router sends traffic to itself. To check the traffic, the Access List Matching Access List technology and QoS traffic routing are used. CIR also writes packets passing through a router to the memory dump, so it is possible to reconstruct their fragments. In this way, we can detect when a packet passes through a router crash.
It is useful to study the report of a critical event that caused a system crash. Here are usually the addresses by which you can track the cause of the problem.
As I said before, we have created a CIR program, Cisco Incident Response, specifically for Cisco, which can be downloaded for free from our website http://cir.recurity-labs.com/ . Here is the version of CIR1.1. This program allows you to analyze the memory dump of the iOS kernel and can be a useful network forensic tool for you. You can use this program to analyze the processes occurring on your own network using Cisco routers and switches.
I note that any attempt to get a system image on the Cisco network will cause the router to reboot. The chance to do this successfully does not exceed 1 per 100,000 attempts. In addition, the data is each time written to different memory stacks, and the addresses of the stacks vary even in the same system image.
Therefore, all hacker exploits for Cisco are based on one fixed iOS image, which must be known in advance before an attack begins. They are suitable for only one of one hundred thousand images of the system. Our program has the ability to track not only existing, but also future generations of malicious exploits.
Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to friends, 30% discount for Habr users on a unique analogue of the entry-level servers that we invented for you:The Truth About VPS (KVM) E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps $ 20 or how to share the server? (Options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).