Many modern computer systems (motherboards in PCs, smartphones, network equipment ...) run under the control of firmware, which, as a rule, is developed for a specific hardware, can use the full range of available hardware resources and has the highest privileges. The protection of firmware should be given special attention, otherwise it will be a real Achilles heel in the security of the system.
A small cycle of articles will be devoted to this issue, we will show weak and strong models of firmware security using real examples.

By the way, the basis of this article is our study of the security of industrial network switches, which I presented at the information security conference for practitioners of ZeroNights 2015 (the report “Modification of firmware of industrial switches”, the presentation is here ).
')

Introduction

First, we will tell about the objects of our research - industrial switches.

The heart of any modern ICS infrastructure is the data transmission environment. In most cases, it is an industrial network such as Industrial Ethernet, i.e. based on the Ethernet technology family. Such a network differs from the usual one, first of all, by using the real-time protocol (for example, Profinet, EtherCAT, Ethernet / IP and others).

An example of an industrial network [picture from here ]:



Various devices (PLCs, HMI panels, control PCs, field devices and others) are connected to the network via industrial-type network switches (switches). Unlike conventional switches, they are less demanding of operating conditions (extended operating temperature range, protection against voltage surges, impact resistance, possibility of mounting on a DIN-rail, etc.) and support industrial network protocols.

Thus, industrial switches are a critical element in the structure of an industrial network. It is very important to properly protect them from being compromised by a potential intruder, because it will allow you to get virtually unlimited control over the process: compromising other devices on the network, intervening and changing data inside various connections between the PLC and SCADA, between the gateways and the PLC, forging data, transmitted to the HMI and logging systems, etc. All this may entail a loss of control by the operator over the real state of the technological process and, as a result, a halt or an accident.

To prevent this from happening:

• first, the location of the switch should minimize the possibility of unauthorized physical access to it;
• secondly, the software and hardware architecture of the switch must have an information security model that takes into account modern threats.

On the last and talk.

Objects of study

For the study, we took one model of managed industrial switches from the two most common manufacturers of industrial network equipment:

• Hirschmann RS20
• Phoenix Contact FL SWITCH MM HS

You can control the operation of these switches through the COM port (both have an RS-232 connector) or at the network level.


When working through the COM port, we are dealing with a console for entering commands:



Access to the console requires a login and password. However, before the authentication screen, several options are available, for example, displaying information about hardware and firmware, updating firmware, etc.


When working with a switch over a network, you can reach (by its IP address) the web interface:



Authentication is required to access it.


Another option for managing the switch at the network level is the Simple Network Management Protocol (SNMP), which also has authentication. However, SNMP support in the switches studied has several drawbacks:

• By default (and this is the most favorite configuration of most network devices), the most insecure version of this protocol is used - SNMP v1;
• in SNMP v1, the default login and password vendor of these devices strongly recommends not to change;
• SNMP v1 and SNMP v2c do not use encryption, which means the protocol is vulnerable to MITM attacks (man in the middle).

First, let's talk about the study of the Hirschmann RS20 switch, then we will designate the features of Phoenix Contact FL SWITCH MM HS in comparison with this switch.

Hirschmann RS20 Industrial Switch Study

Firmware

First you need to get the firmware, understand what the processor is doing, and evaluate the possibility of connecting the programmer / debugger. To do this, we looked inside and saw:

1. CPU Digi NET + ARM NS9360B-0-I155, 32-bit ARM9, without internal memory (therefore, the firmware must be stored in external memory);
2. Micron MT48LC8M16A2 SDRAM RAM at 16 MB;
3. Intel flash memory 28F640JD3D75 8 MB (most likely, the firmware here);
4. CPLD Marvell 88E6095F-LG01, performs the function of an Ethernet switch, has an internal configuration memory.

Chip flash memory is made in the case of BGA. If we drop it, it will be very difficult to solder it back without an infrared soldering station. Therefore, the firmware (version 8.0.07) for this switch has been merged from the open Hirschmann ftp server .

One of the files in the downloaded archive is the rsL2E.bin binary (about 4 MB), this is the firmware image. Having rummaged in it, we saw that it consists of two modules. Each module has a header (their structure is the same), as well as a compressed body.




The first module is compressed using the zlib algorithm (RFC 1951). That is where the executable code is stored. When unpacked, the module weighs approximately 7 MB.

The second module is compressed using the gzip algorithm (RFC 1952), it contains a pack200 archive, after unpacking which you get a JAR (Java Archive) file, which is an implementation of the device’s web interface. After unpacking, the module weighs approximately 3 MB. By the way, this file is transmitted to the host (control PC) when accessing the switch by its IP address (that is, when trying to access the device’s web interface) and is executed on it (on the host).

It can be seen that the total amount of unpacked modules is 10 MB, and the amount of flash memory on the system board of the device is 8 MB. Therefore, we assumed that the firmware image is stored in a packed form. So, there must be a bootloader that unpacks and launches this firmware. But about the loader later.


The size of each module header is 256 bytes. The structure of it is not documented anywhere, so I had to disassemble the method of scientific tyke. Fields that caught our attention:

• module signature;
• type of module;
• module size;
• module checksum (CRC32);
• offset to the end of the file (eof offset);
• module checksum (repeat) (CRC32);
• header checksum (CRC32).

Based on this, it can be concluded that there is a control of the integrity of the firmware. But the control of authenticity, usually a digital signature (digital signature), is absent.

The ability to falsify these firmwares must be checked, for this we wrote a small script that allows you to quickly remove both modules from the firmware image, and, after making changes to them, collect them back into the finished image:




But before you make changes, you need to think. And messed up.


It turned out that inside the firmware there is a code of RTOS (real-time operating system) VxWorks, with a rather old version 5.4.2 (around 1998). At the time of this writing, the latest version is 7.

In the Chinese "Internet" we found the source for VxWorks 5.5, which greatly facilitated the identification of various operating system functions, libc procedures and other things. Anyone who is faced with the task of analyzing VxWorks code will probably think now: “Look at the end, VxWorks images always have a symbol table!”. It is, but in this form it is almost useless:




VxWorks is a modular product, and the composition of the modules is determined by the customer when they buy source codes. In this case, in addition to the basic components, we saw the modules of the web server (EmWeb), SnmpOverHttp (yes, in the dropped traffic, we saw SNMP packets inside HTTP requests) and others.
The vendor definitely took care of the filling.

But the security is not very. Here, the linear address space, the execution of the code can be carried out from anywhere in the memory, there is also no protection against overflows on the stack, etc. Elementary protection mechanisms against exploitation of binary vulnerabilities are completely absent. Maybe they are not needed? But after all the registered vulnerabilities for this version of VxWorks are complete :

1. CVE-2015-3963 spoof TCP sessions;
2. CVE-2010-2968 brute-force;
3. CVE-2010-2967 obtain access;
4. CVE-2010-2966 obtain access;
5. CVE-2010-2965 RCE;
6. CVE-2008-2476 DoS;
7. ...

Choose for every taste: DoS (Denial of Service), RCE (Remote Code Execution), opportunities for selection (brute-force), opportunities for circumventing authorization. The list is not complete.

But do not be upset! Indeed, during the analysis of the firmware, very interesting code fragments were found:

• SNMP request handlers;
• console command handlers;
• reading and rewriting flash memory;
• read and overwrite CPLD Marvell configuration memory!

Povorevsili, now you can pohahit.

To do this, you should correctly choose a place to introduce the code: it should not be allowed to interfere with the normal functioning of the switch, and it is best if it is called up on demand.

The obvious choice was one of the handlers of console commands. More specifically, the “logout” command handler. In its place in the firmware, we wrote a code that reads the memory of a given size at a given address (everything is set in variables at the end of the inserted fragment), and outputs the captured dump to the COM port:




We made a ready image, updated the switch with it (it is possible to update the firmware through the COM port and through the web interface). Success! Indeed, after entering the logout command, instead of logging out, a memory dump was now output:




Thereby, the possibility of forging firmware was confirmed.

But how can an attacker embed his firmware into a switch that is actually used at an industrial facility?


Consider the possibility of updating via the COM port (i.e., via the console interface). As already mentioned, to upgrade the firmware of the switch, you do not need to know your login and password, but you must have physical access to the device (to plug into RS-232). This option is hard to believe if the device is already in use at the facility.

However, the switch from the factory does not immediately reach the industrial facility. In the delivery scheme to the customer there are always intermediate organizations that have the ability, as a free bonus, to download non-original firmware to the switch.

Another option (although this is already the distribution vector): you can infect the firmware of the switch from a previously compromised control PC that is already connected to the switches via the COM port.


Remotely modifying the firmware of the switch is more difficult, but the attack surface is much wider. Recall that authentication is required to access the switch web interface. What can be done? Try:

• enter default login and password;
• pick up taking into account known restrictions on the password. Fortunately, there is no protection against brute-force attacks;
• exploit binary vulnerabilities (known or honestly find one's own), again, there is no protection against exploitation of binary vulnerabilities.

By the way, remotely - this is not only from a PC connected to the switch. And also from one of the field devices, which happens to be in unguarded territory. So it must be taken care of!


The conclusion is the following. A potential attacker, when introducing his code into the switch firmware, has the ability to:

• execute its code on the switch CPU (using the whole range of its hardware resources);
• execute its code on the side of the host (control PC) or any other client PC that accidentally called on the IP address of the switch. Of course, this code will be executed in a Java machine, but, as you know, it is not a security bulwark.

Note that a compromised switch can be “cured” by a regular firmware update procedure. The main thing is that you are sure of the authenticity of the new image.


In any case, this immediately leads to the question of the possibility of fixing on the device so that the changes made to the firmware would “experience” any updates. And then we remembered the bootloader, the dump of which we did not have.

Loader

Hoping that the bootloader left a trace in the RAM, we dumped (using the above code snippet) it in different places, but did not find anything. And we decided to try to get it from the flash memory of the switch, pulling the previously found procedures for reading flash memory. And success again.

So, the contents of flash memory can be divided into three regions:

1. Bootblock, first 80000h bytes. This is the boot area, here is stored the bootloader, which consists of three parts:
   
   • start code;
   • main code (compressed by Huffman algorithm);
   • header (the structure is identical to the previously described structure of the header of the firmware module);
2. Application firmware, the largest area, stores the image of the firmware in a packaged form, as expected;
3. Storage, storage area for service information, configurations, logs, etc.

The bootloader code allowed us to reconstruct the early stages of the switch boot process.

At the start, the first 4 KB of flash memory (which is just the size of the start code) are projected into the CPU address space at address 0. From this address, execution begins.
The task of the starting part of the bootloader is to configure the memory card, unpack the main part of the bootloader into memory and transfer control to it.

The main loader code should:

1. Initialize CPU hardware resources;
2. Configure the interrupt model;
3. Unpack the firmware into memory and transfer control to it.

Immediately comes to mind a way to modify the bootloader. The switch is updated with a modified version of the firmware, which, using standard procedures for working with flash memory, will rewrite the bootblock area.



In the future, the modified bootloader will be able to make changes to the unpacked firmware image at each boot. A switch thus compromised is not so easy to cure.




Is it possible to “cure” a switch with regular means? To do this, you need to update on the original and firmware, and bootloader.

In the console interface, it is not possible to update the bootloader, although procedures have been found in the firmware code that are designed to update the bootblock area in the same way as the firmware is updated. But these functions are not called anywhere, so this is an undocumented debugging functionality.

But in the web interface of the switch a fully-fledged option was found for updating the bootloader:




The only question is what to update? And nothing!

In the archive with the firmware image for many models of their devices, Hirschmann does not attach the boot image. And the investigated model among them.

Get the image of the bootloader by contacting tech support, too, will not work. We tried, and got the answer, that the bootloader images are not distributed. The only thing that was possible to achieve from them was the proposal to send the device to them using the RMA form. However, they will strongly discourage you from doing this, sincerely considering this process to be useless.

Thus, a potential attacker has the opportunity, through a modification of the firmware, to be fixed in the device loader. And it is almost impossible to get it out of there using regular means.

Is it possible to dig even deeper?

At the moment we have no answer to this question, but the idea will be worth it.

Recall that this device has another executable environment, with its own, in some way, firmware. This is an FPGA (programmable logic integrated circuit), such as CPLD from Marvell with internal non-volatile configuration memory.

Programs for them are written in the language of the description of logic (VHDL, Verilog - the most common). Recall that in the switch firmware code, we found functions for reading and rewriting this configuration memory.

Research industrial switch Phoenix Contact FL SWITCH MM HS

In the study of this switch, we went the same way, only faster. What is interesting inside:

1. CPU PMC RM5231A, 32-bit MIPS IV, without internal memory;
2. Micron MT48LC8M16A2 SDRAM 16 MB RAM, two pieces;
3. Intel flash memory of unknown model;
4. Chipset Galileo GT-64115.

We downloaded the firmware image for this switch from the vendor’s official website and dismantled its structure in the same way. It has a header and a zlib compressed body (RFC 1951) body:




Primary fields in the header structure:

• signature;
• header checksum (ADLER32);
• checksum of the unpacked body (ADLER32);
• unpacked body size;
• packed body checksum (ADLER32);
• packed body size.

Here, too, there is no control over the authenticity of the firmware, we have experimentally verified this.




VxWorks is also here, although a slightly newer version 6.1 has the same problems: the lack of protection mechanisms against exploitation of binary vulnerabilities and a pack of registered CVEs.

You can update the firmware in the same ways:

• via COM port, authentication is also not required
• via web interface. To access it, you do not need to enter a login and password, but when making changes to the settings (or when updating the firmware) you need to know the password.

Speaking of password. According to the documentation, there is an engineering password, in case the current password is lost (well, or never acquired, if you are a hacker). To do this, you need to contact technical support by providing them with the MAC address and serial number of the switch.
This suggests that there is an algorithm for converting these numbers into an engineering password.

This switch also has a bootloader that can be overwritten. And it is experimentally proven. There are no regular features for updating the bootloader at all.

Conclusion

As a result of the study, the following shortcomings in the architecture of industrial switches were revealed:

1. The ability to illegally update the firmware (for some standard procedures, authentication is not required);
2. The ability to fake the image of the device firmware (later the boot loader, the CPLD firmware ...) because there is no control of the authenticity of the code;
3. There are no mechanisms for exploiting binary vulnerabilities, which opens up additional possibilities for remote execution of code on the switch.

During the study, we showed the ability to fake firmware switches, as well as the possibility of fixing modifications in the bootloader.


In short, pahacheno.

A real example of the correct, in our opinion, firmware security model will be discussed in the next article.