How Huawei secret key got into Cisco router firmware

The developers of the IoT Inspector scanner periodically conduct a routine check of firmware from different manufacturers. Sometimes they find vulnerabilities in Internet of Things devices. But the latest find is especially amazing. In the firmware of the Cisco SG250 router, the scanner found several certificates and the corresponding secret key of the employee of Futurewei Technologies, the American subsidiary of Huawei Technologies, named Garry Wu with the email address gary.wu1@huawei.com .

But how did they get into the firmware of the Cisco router?

The location of the files in question (/root/.ssh/) was also strange. This directory is usually intended for SSH keys, not certificates.
')


The developers manually checked the results of the auto scanner. They were reliable, manual analysis confirmed the automated results.



Given the political controversy surrounding Huawei, the employees did not want to speculate on this topic and immediately transferred information to Cisco. There, the security department confirmed the results and began an internal investigation. It took them only a few days to find out the reasons and share the details.

As it turned out, the certificates and private key in question were part of the free OpenDaylight GitHub package, which is used in some Cisco products. The issue affected all Cisco 250/350 / 350X / 550X series routers. The developers used certificates to test the Cisco FindIT function to proactively manage the network of routers with automatic finding of all devices in the network, configuration via a single interface, etc. Certificates were found in product firmware due to a simple oversight.

Cisco immediately released the firmware without these certificates, and on July 3, published security recommendations . In addition, Cisco fixed other problems detected by the scanner: empty password hashes, unnecessary dependencies, and various vulnerabilities in dependencies.

There was no security threat here. About the mythical "backdoors of Huawei" is not in question. According to Cisco, no attack vectors have been defined, since certificates are not actually used in the final firmware versions.

But still there are two problems here.

First, developers often, for convenience or by inadvertently, place secret information into program code, which is then laid out in open access. For example, in March 2019, researchers at the University of North Carolina scanned GitHub and found that over 100,000 repositories contain API keys, tokens, and cryptographic keys. In addition, thousands of new, unique “secrets” fall on GitHub every day. General statistics on found secret objects shows that most often Google API keys are in open access. Also, RSA private keys and Google OAuth IDs are common.



The study showed that an attacker, even with minimal resources, could compromise many GitHub users and find a lot of secret keys. To combat this problem, GitHub developed the Token Scanning token scanning function, which scans repositories, searches for tokens and notifies service providers of information leaks.

The second problem is that programs increasingly connect projects on the same GitHub as external dependencies of Open Source, without checking them accordingly. It turns out that developers blindly trust someone else's code, which no one can follow. The problem is described, for example, in the article of Russ Cox “Our problem with dependencies”, which is translated in Habré.

---