NVRAM device in UEFI-compatible firmware, part one

Hello, dear readers. Once upon a time, almost 3 years ago, I wrote a couple of articles about data formats used in UEFI-compatible firmware. Since then, little has changed in these formats, so I will not write about them again. However, in those articles there was a rather serious gap - there was no mention of NVRAM and the formats used for its storage, since then NVRAM was simply uninteresting to me, because the same data can be obtained from the UEFI Shell on a working system with just one dmpstore command.
After three years, it turned out that the NVRAM storage can collapse for various reasons, and most often this event leads to a “brick”, i.e. use the above command is no longer possible, but the data (or what remains of them) must be got. Having collected a couple of collapsed NVRAMs manually in a Hex editor, I said “ enough to endure it! ”, Added support for parsing NVRAM formats in UEFITool NE , and decided to write a series of articles about these formats in hot pursuit and fresh memory.
In the first part, let's talk about what this NVRAM is all about, and consider the VSS format and its variations. If interested - welcome under cat.

Denial of responsibility

Not daring to violate the already established tradition, I have to write that all the information that you can gather here can lead to the destruction of your NVRAM, firmware, system and faith in humanity if you are careless or misused. The author is not responsible for anything at all, use this knowledge at your own peril and risk, do exercises, eat well and sleep more.

All the information described in this article was obtained using reverse engineering of many different images of UEFI-compatible firmware, and therefore do not pretend to be one hundred percent complete or true. If you find an error - report it in the comments, I will be glad to fix it.

Introduction

To begin with, what this NVRAM is all about and why the authors of the UEFI specification suddenly needed it, taking into account that everyone used to quietly store their CMOS SRAM settings on a battery and did not buzz. I have already talked a little about the “logical” level of NVRAM, and here I will try to tell you more about the “physical”.
So, NVRAM is such a special data area in which those UEFI variables are stored that have the Non-Volatile attribute set. The most popular variables of this kind are Setup , which stores most of the current settings from BIOS Setup, BootXXXX / BootOrder / BootNext , which controls the boot order, PK / KEK / db / dbx / dbt , responsible for the SecureBoot operation, MonotonicCounter , which protects from replay -attack to the previous five, and many others, a specific list depends on the vendor, the board model and the version of its firmware.
')
Most often, NVRAM is located on the same SPI chip as the executable firmware code, for one simple and banal reason - it is practically free (for 100-200 KB on an 8 MB chip can be found almost always, and a separate CMOS SRAM chip on 128 KB is very tangible money). The free nature of this leads to several very serious risks:

1. If there is an error in the NVRAM driver, then it can destroy not only its data, but also the data of its neighbors, including the volume in which the code is stored, then after the reboot, the machine will get a stake, and recovering it from this state will be very difficult.
2. Each entry in NVRAM (and they are usually made several at every power up and every reboot) reduces the life of the SPI chip, and under certain conditions (for example, at constantly high temperatures, which is not uncommon for industrial PCs) after 3-5 years This is fully developed and the system begins to behave very strange. At the same time, the manufacturers of SPI-chips of the 25th series do not provide any SMART, EXT_CSD or automatic wear-out leveling analogs, and I have already seen systems a couple of times on which the chip simply “tired” until it was completely inoperable and had to be changed.
3. It is impossible to reset a destroyed or incorrect NVRAM with a jumper or by removing the battery, you need to erase it with an external to the storage device of the SPI device. Some manufacturers mimic the behavior of the CLEAR_CMOS jumper users with a special DXE driver, storing CMOS SRAM (which is still there, but now it is much smaller, since only the watch and a pair of flags are stored in it) of the NVRAM_IS_VALID flag. If at the next boot this flag is cleared, the default values ​​for variables like Setup are restored. Unfortunately, very often it does not help, because Before this driver was loaded, there was an entire PEI phase, in which there were also modules with requests for NVRAM, and if the requests could not be satisfied, then nothing would be restored, because the download will stop earlier.

NVRAM Requirements

When implementing the “physical” NVRAM level, firmware manufacturers had to solve many questions: how to provide quick access to variables for reading (they are read quite actively during loading), how to reduce the load on flash memory while writing, how to store variables in such a way that duplicate data that is common to several variables (vendor GUIDs, for example), how to recover at least some of the data after a failure, and so on. At the same time, the NVRAM data storage format proposed by Intel with the release of the EFI 1.10 standard turned out to be simple, but not all of the above requirements, plus its format was not described in the UEFI PI specification, i.e. The choice of NVRAM implementation was left to the end vendors.

As a result, instead of a single FFSv2 format, which then received an extended header and a couple of controversial fields in ZeroVector , but remained exactly the standard, vendors managed to implement three fundamentally different formats for NVRAM, which makes its analysis a very exciting activity.

What are the formats

Before talking about formats, let's talk a little about their names. Each vendor, following the long tradition of calling their country "country" or "land", and its people - "people", calls its format "storage format NVRAM", which somewhat prevents them from distinguishing. But we were lucky: NVRAM is usually stored inside a special volume with a relatively arbitrary structure, then the storage headers have signatures, and these signatures are different for each format. I will call them by signatures, although this terminology has not yet been settled.

The first historically and prevalence turned out to be the Intel-proposed VSS format at the dawn of EFI development, which in UEFI 2.3.1C standard was expanded to support the protected variables used to implement SecureBoot, and also received a couple of extensions from Apple used only in their firmware. An FTW block can be stored next to the VSS data, the data from which can help restore NVRAM in the event of an abnormally incomplete recording (remember that “computer power can be turned off” at any second). After the implementation of SecureBoot, it was necessary to store the default values ​​for its variables, for which some vendors added to the same format an FDC block (also named after the signature), where these “defaults” are stored.

Almost immediately it turned out that storing NVRAM only in VSS format is not at all necessary, so one of the vendors (I don’t know exactly who was first, I think it was Phoenix) implemented the EVSA format, in which GUID deduplication and variable names, but the FTW capabilities are gone. This format has not received much distribution, but sometimes still no, no, yes, it is found in the old firmware of the UEFI 2.1 era. For their repositories, EVSA uses the same primary and secondary NVRAM volumes as VSS, so the analysis of the structure of these volumes, as I have already said, is very exciting.

Apple went even further, and added two more data blocks to the same long-suffering volumes - SVS , the format of which coincides with the usual VSS with signature accuracy, and Fsys , the format of which was invented by Apple from scratch.

The latest format in our list is NVAR , developed by AMI and used by them from the very first implementations of Aptio4, has since gone through two updates, one of which added a checksum for data stored in a variable, and the second - support for secure SecureBoot variables. The format itself is quite interesting, uses GUID deduplication, optimizes the character size in variable names (which, by specification, in UCS2 encoding), if all of them fit into a single-byte encoding, are relatively resistant to failures, but need periodic garbage collection. Unfortunately, the updates did not affect the best way, and his analysis after them became more complicated, and with it the likelihood of errors increased, so it is not clear whether AMI won anything from the decision not to use VSS or not.

The list turned out to be quite impressive, but it usually happens when the specification gives the vendors too much freedom of choice, and they cynically use this freedom.

VSS format and variations

The NVRAM data in all UEFI-compatible firmware I have seen, except those based on the AMI code (which I will discuss in the section dedicated to the NVAR format), is stored in one or several GUID volumes FFF12B8D-7696-4C8B-A985-2747075B4F50 (also EFI_SYSTEM_NV_DTD I call it “main”), either with GUID 00504624-8A59-4EEB-BD0F-6B36E96128E0 (I call it “optional”).
Both volumes have a sparse structure, so you have to look at their bytes per byte in search of storage and block signatures. The VSS repository header looks like this:

struct VSS_VARIABLE_STORE_HEADER { UINT32 Signature; //  UINT32 Size; //       UINT8 Format; // ,   ,       (0x5A) UINT8 State; // ,   ,        (0xFE) UINT16 Unknown; //  ,     Apple SVS UINT32 : 32; //   }; 

Not everyone still knows how to parse the structures of the C language on the fly, so it makes sense to show the same structure in the screenshot:

It is easy to see that we have before us the VSS storage header with the corresponding signature , with a total size of 0xFFB8 bytes, correctly formatted and with correct data .
Apple sometimes uses the same title, but with a different signature - $ SVS . Why this is done - I do not know, think different , apparently.
Immediately after the repository header, the variables stored in it begin. They are located one after another, and on all architectures except IA64 (also known as Itanium), for which the requirement to align the beginning of variables along the eight-byte boundary is mentioned, but I simply do not have firmware images for this architecture to check this statement.

The formats of variables for the ten-year history of VSS have accumulated three pieces: the old one, used before UEFI 2.3.1C, its extension from Apple with an additional field for the CRC32, and a new one, which was needed to support SecureBoot. Perhaps there are some others, but I haven’t yet managed to find images with them, maybe the readers will succeed.

Standard

This format was widely used by almost all manufacturers of UEFI-compatible firmware, except AMI, for about seven years, until the introduction of SecureBoot was required. The heading of the “standard” variable looks like this:

 struct VSS_VARIABLE_HEADER { UINT16 StartId; //    (0xAA 0x55) UINT8 State; //   UINT8 : 8; //   UINT32 Attributes; //   UINT32 NameSize; //   ,    0-   UCS2 UINT32 DataSize; //  ,    EFI_GUID VendorGuid; // GUID  }; 

This time on the screenshot you can show several variables at once:

More precisely, one and a half: PchInit and part of the Setup . They have a status of 0x7F (VARIABLE_HEADER_VALID), attributes 0x07 ( NV + BS - RT ), the length of the name 0x10 and 0x0C, the length of the data 0x04 and 0x2B0, and the GUID E6C2F70A-B604-4877-85BA-DEEC89E117EB and 4DFBBA IBABA-560-A560A-B604-4877-85BA-DEEC89E117EB and 4DFBBA IB-API. C41CC5AD7D5D respectively.

If you do not feel like disassembling manually, you can use the latest alpha version of UEFITool NE, from which the NVRAM volume from the screenshots above looks like this:

Apple CRC

About a couple of years ago, Apple decided that their variables lacked a checksum, and therefore added another field to the header above, in which the CRC32 checksum of the variable data block is stored. This format Apple uses to this day, and most likely will continue to use in the future. Its headline looks like this:

 struct VSS_APPLE_VARIABLE_HEADER { UINT16 StartId; //    (0xAA 0x55) UINT8 State; //   UINT8 : 8; //   UINT32 Attributes; //   UINT32 NameSize; //   ,    0-   UCS2 UINT32 DataSize; //  ,    EFI_GUID VendorGuid; // GUID  UINT32 DataCrc32; // CRC32-   }; 

I will not apply screenshots, everything is completely analogous, I will only say that Apple uses the additional attribute 0x80000000 (CRC_USED) to distinguish its title from the standard one.

Authenticated

After the UEFI Forum decided to use NVRAM to store keys used by the SecureBoot technology , it was necessary to refine the format. New variables received the following title:

 struct VSS_AUTH_VARIABLE_HEADER { UINT16 StartId; //    (0xAA 0x55) UINT8 State; //   UINT8 : 8; //   UINT32 Attributes; //   UINT64 MonotonicCounter; // ,   replay- EFI_TIME Timestamp; //  ,     replay- UINT32 PubKeyIndex; //     ,  0,      UINT32 NameSize; //   ,    0-   UCS2 UINT32 DataSize; //  ,    EFI_GUID VendorGuid; // GUID  }; 

In the screenshot, this variable looks like this:

The marker is the same as for ordinary variables, the state in this case is 0x3F (VARIABLE_ADDED), the attributes are 0x27 (BS + NV + RT + TA ), the counter is not used, but the time stamp is used in the EFI_TIME format, the index in the public key database is also not involved, the name size is 0x08, the data size is 0x64D, the GUID is D719B2CB-3D3A-4596-A3BC-DAD00E67656F, and this name is dbx.

In UEFITool, the same variable looks like this:

Conclusion

Well, the VSS formats are more or less sorted out, next time we will talk about the Fsys, EVSA and NVAR formats, as well as the various data blocks that can be found next to the main NVRAM.
I hope that you liked the first part, thank you very much for your attention and see you in the second part.