Micro DB for configuration on microcontroller
Many software developers for microcontrollers are faced with the problem of storing configuration data. These can be calibration coefficients of the measurement algorithm or simply the last menu item selected by the user. For a microcontroller with the ability to write to its own flash-memory, the solution seems simple - erase the segment and write everything you need there. However, if it is also necessary to provide fault tolerance in relation to power off at an arbitrary moment, the task becomes nontrivial - in fact, it is necessary to implement a small database with a mechanism for ensuring the atomicity of write operations and recovery after failures. The solution of this problem for microcontrollers of the MSP430 family is under the cut. By the amount of resources used, it is suitable even for the youngest members of this family - with a RAM size of 256 bytes and falsh memory of 8Kb. As a bonus, there is a command line interface (via UART) for reading and writing configurations.
We restrict ourselves to storing two-byte integers and strings with a length of no more than 15 characters, which is quite enough for most applications. If desired, this set is easy to expand by minor code modifications. Stored data will be identified by name and possibly one or two indexes. We will also limit the maximum index to the 15th, while the integer parameters can have 2 indexes, but only one line - the second index will be taken by the length of the line.
The first thing that comes to mind is to store the name first, then the data. But in our case, this approach is categorically ineffective. The fact is that we cannot simply rewrite old data. We will have to write a new value to a new place and only then to figure out which of them is the most recent. This means that we may have to write down the same name many times. Therefore, we will store the names separately by entering a new entity - the type of configuration data. Type is a name plus a description of what can be stored under this name. Namely, the base type is an integer or a string, and the dimension of the indices is 0, 1 or 2 for integers, 0 or 1 for strings. In addition, the default type will be stored in the type descriptor, which we will return when reading data that has not yet been written. The type table will be placed in flash memory at compile time and will never change. When saving data, we simply refer to the type, indicating its index in the table, for which we will need one byte. In another byte, we will pack 2 indexes of an instance of this type, or an index and string length. Then the actual data will be recorded.
Before you write something in flash-memory, it must be erased. Erasing occurs in 512 byte segments. This means that we will need at least 2 of these segments to store data. One will store data until we erase the second. If there is a lot of data, take 2 working areas from N segments. While one stores data, the second can be erased. In general, the algorithm is as follows - we write data sequentially to the workspace, after the place finishes, we delete the nonworking region, save the latest versions of all the data from the workspace (snapshot), and then change the workspace. In this case, we are left with the only problem - how then to choose the workspace that was recorded last?
')
The standard solution to finding the latest version of an object is to assign sequence numbers to versions. It works well if these numbers are long enough so as not to be repeated throughout the entire life of the program. In our case, it is difficult for us to afford numbers longer than 2 bytes. This means that the actual numbers will be placed in a circle with a circle length of 65536. If our copies of the configuration data occupy less than half of such a circle, then they can be ordered according to the degree of freshness, and if there are more, then no longer. This means that when copying data to snapshot, we are forced to update their sequence numbers. But then, if the copy operation is interrupted in the middle, there will be complete confusion - part of the data will have more recent copies, and some will not, and we will not be able to simply select one of the two work areas. But this is still half the trouble. The most annoying thing is how the power is turned off at any time.
To understand the scale of the disaster, the author wrote an automatic test that implements 2 models of emergency interruption of the program - a reset from the interrupt handler and interruptions in the power circuit. The first method did not lead to something that even a simple implementation based on sequence numbers could not handle. This is not surprising if we consider that when performing erase and write operations in flash, the execution of the program from the same flash just stops, and we have no chance to interrupt these operations in the middle. Therefore, a second, more radical method was implemented. The power to the microcontroller was supplied through a resistor in the 510th, and one of the universal legs was connected to ground. To simulate a power failure, this leg was switched on to the output, and a high level was applied to it. As a result, the current consumption sharply increased, and the supply voltage fell below the permissible. As a result, it turned out that the worst thing that could happen was the incomplete erasure of a segment, as a result of which its contents could become anything. As a result, the use of checksums to detect this situation came to the fore, and it was decided to abandon the use of sequence numbers. The CRC16 checksum is written after the data, so for each record we have 4 bytes of additional information - 2 bytes of the header and 2 bytes of the checksum. However, checksums alone do not solve the problem of choosing a workspace after a failure.
In order to mark the workspace, it was decided to simply write into it a special 'status tag'. Its format is similar to the data format with the only difference that the data is not there, and the type index is equal to the maximum possible, i.e. 255. Since it is possible that we will have 2 labeled areas, we will introduce another label, which will be marked with an area in which we are no longer going to record anything. We will call the first label opening, and the second closing. When creating a snapshot, we first write the closing label to the old workspace, and then create a new snapshot in the new one and write the opening label to it. We will complete the switching of the working area by writing the final label to the old working area. The meaning of this action will become clear below.
In the event of a power outage during recording, latency errors may occur. If the charge injection into the floating gate of the flash-memory element is not completed, then for some time the correct data can be read from it, and then the wrong data will be read. The appearance of such an error in the middle of the workspace will have fatal consequences. However, there is an easy way to avoid the continuation of the record after the data that could have been interrupted. This is the purpose of the final label. Since it is written last in an area that is no longer working, if we read the final label, the recording of the label that opens the working area is guaranteed not to be interrupted. We call this combination of the opening label in the workspace and the final one in the non-working state a stable state. If at the start we see an unstable state, then before writing new data, we create snapshots and change the workspace, thereby bringing the system into a stable state.
Testing by recording 5 million values with a simulation of power interruptions 20,000 times did not reveal any problems, after which the experiment was stopped to avoid the full development of the flash-memory resource.
Lies here . The project for IAR is designed for MSP430G2553, the author used the latest version of MSP-EXP430G2 LaunchPad as a hardware platform. The project implements an automatic test, but you can easily adapt it to your own tasks. It also has a command line interface (via UART) for accessing the configuration. The types command prints the type list, the cfg command prints the current contents of the repository, the set command updates the configuration. The help command prints help on commands.
Data format
We restrict ourselves to storing two-byte integers and strings with a length of no more than 15 characters, which is quite enough for most applications. If desired, this set is easy to expand by minor code modifications. Stored data will be identified by name and possibly one or two indexes. We will also limit the maximum index to the 15th, while the integer parameters can have 2 indexes, but only one line - the second index will be taken by the length of the line.
The first thing that comes to mind is to store the name first, then the data. But in our case, this approach is categorically ineffective. The fact is that we cannot simply rewrite old data. We will have to write a new value to a new place and only then to figure out which of them is the most recent. This means that we may have to write down the same name many times. Therefore, we will store the names separately by entering a new entity - the type of configuration data. Type is a name plus a description of what can be stored under this name. Namely, the base type is an integer or a string, and the dimension of the indices is 0, 1 or 2 for integers, 0 or 1 for strings. In addition, the default type will be stored in the type descriptor, which we will return when reading data that has not yet been written. The type table will be placed in flash memory at compile time and will never change. When saving data, we simply refer to the type, indicating its index in the table, for which we will need one byte. In another byte, we will pack 2 indexes of an instance of this type, or an index and string length. Then the actual data will be recorded.
Memory organization
Before you write something in flash-memory, it must be erased. Erasing occurs in 512 byte segments. This means that we will need at least 2 of these segments to store data. One will store data until we erase the second. If there is a lot of data, take 2 working areas from N segments. While one stores data, the second can be erased. In general, the algorithm is as follows - we write data sequentially to the workspace, after the place finishes, we delete the nonworking region, save the latest versions of all the data from the workspace (snapshot), and then change the workspace. In this case, we are left with the only problem - how then to choose the workspace that was recorded last?
')
Sequence numbers
The standard solution to finding the latest version of an object is to assign sequence numbers to versions. It works well if these numbers are long enough so as not to be repeated throughout the entire life of the program. In our case, it is difficult for us to afford numbers longer than 2 bytes. This means that the actual numbers will be placed in a circle with a circle length of 65536. If our copies of the configuration data occupy less than half of such a circle, then they can be ordered according to the degree of freshness, and if there are more, then no longer. This means that when copying data to snapshot, we are forced to update their sequence numbers. But then, if the copy operation is interrupted in the middle, there will be complete confusion - part of the data will have more recent copies, and some will not, and we will not be able to simply select one of the two work areas. But this is still half the trouble. The most annoying thing is how the power is turned off at any time.
Checksums
To understand the scale of the disaster, the author wrote an automatic test that implements 2 models of emergency interruption of the program - a reset from the interrupt handler and interruptions in the power circuit. The first method did not lead to something that even a simple implementation based on sequence numbers could not handle. This is not surprising if we consider that when performing erase and write operations in flash, the execution of the program from the same flash just stops, and we have no chance to interrupt these operations in the middle. Therefore, a second, more radical method was implemented. The power to the microcontroller was supplied through a resistor in the 510th, and one of the universal legs was connected to ground. To simulate a power failure, this leg was switched on to the output, and a high level was applied to it. As a result, the current consumption sharply increased, and the supply voltage fell below the permissible. As a result, it turned out that the worst thing that could happen was the incomplete erasure of a segment, as a result of which its contents could become anything. As a result, the use of checksums to detect this situation came to the fore, and it was decided to abandon the use of sequence numbers. The CRC16 checksum is written after the data, so for each record we have 4 bytes of additional information - 2 bytes of the header and 2 bytes of the checksum. However, checksums alone do not solve the problem of choosing a workspace after a failure.
Status tags
In order to mark the workspace, it was decided to simply write into it a special 'status tag'. Its format is similar to the data format with the only difference that the data is not there, and the type index is equal to the maximum possible, i.e. 255. Since it is possible that we will have 2 labeled areas, we will introduce another label, which will be marked with an area in which we are no longer going to record anything. We will call the first label opening, and the second closing. When creating a snapshot, we first write the closing label to the old workspace, and then create a new snapshot in the new one and write the opening label to it. We will complete the switching of the working area by writing the final label to the old working area. The meaning of this action will become clear below.
Latent errors
In the event of a power outage during recording, latency errors may occur. If the charge injection into the floating gate of the flash-memory element is not completed, then for some time the correct data can be read from it, and then the wrong data will be read. The appearance of such an error in the middle of the workspace will have fatal consequences. However, there is an easy way to avoid the continuation of the record after the data that could have been interrupted. This is the purpose of the final label. Since it is written last in an area that is no longer working, if we read the final label, the recording of the label that opens the working area is guaranteed not to be interrupted. We call this combination of the opening label in the workspace and the final one in the non-working state a stable state. If at the start we see an unstable state, then before writing new data, we create snapshots and change the workspace, thereby bringing the system into a stable state.
Using
Create a type table with default values:
#include "cfg_types.h" const struct cfg_type g_cfg_types[] = { CFG_STR("", ""), CFG_INT("", 1), CFG_INT_I("", 0), ... CFG_TYPE_END }; unsigned g_cfg_ntypes = CFG_NTYPES; Create a configuration repository:
#include "config.h" #pragma data_alignment=FLASH_SEG_SZ static __no_init const char cfg_storage[2][CFG_BUFF_SIZE] @ "CODE"; struct config g_cfg = {{ {cfg_storage[0]}, {cfg_storage[1]} }}; We initialize it:
#include "config.h" extern struct config g_cfg; cfg_init(&g_cfg); We read data:
const char* name = cfg_get_str(&g_cfg, get_cfg_type("")); const struct cfg_type* cnt_t = get_cfg_type(""); int cnt = cfg_get_val(&g_cfg, cnt_t); Update the data:
cfg_put_val(&g_cfg, cnt_t, cnt+1); Test results
Testing by recording 5 million values with a simulation of power interruptions 20,000 times did not reveal any problems, after which the experiment was stopped to avoid the full development of the flash-memory resource.
Source
Lies here . The project for IAR is designed for MSP430G2553, the author used the latest version of MSP-EXP430G2 LaunchPad as a hardware platform. The project implements an automatic test, but you can easily adapt it to your own tasks. It also has a command line interface (via UART) for accessing the configuration. The types command prints the type list, the cfg command prints the current contents of the repository, the set command updates the configuration. The help command prints help on commands.
Source: https://habr.com/ru/post/244773/
All Articles