Streaming Protocol for PSP1N Microcontrollers

Formulation of the problem

When developing the next device on the microcontroller, I ran into a situation where continuous registration of a large amount of data was required. The device had to save a dataset consisting of a time stamp and measurement results of six ADCs from 100 times per second to an SD card (let's call this dataset - package), and then analyze these data on a computer in the form of graphs. It was also necessary in parallel with the recording of data on the SD card, transfer them to the UART. This data should occupy as little space as possible, since there is a lot of data. At the same time, it was necessary to somehow separate these packets, because the data went in a continuous stream. Rummaging on the Internet did not find anything good, so I decided to create my own protocol and libraries for it.

And then he appeared - Packet streaming protocol (PSP1N)

As a result of some deliberation, it was decided the following: in the protocol, data will be transmitted in packets consisting of N bytes, where the first bit of each byte is allocated to the start bit flag to synchronize the packets (hence the name of the protocol), the remaining seven bits are allocated for data. The sequence and size of the data is known in advance.

Example:

Select 32 bits for the time stamp, 60 bits for ADC measurements (6 channels of 10 bits each), total 92 bits are required. Since we can transmit 7 bits of useful data in a byte, the packet will consist of 14 bytes (92 bits / 7bit = 13.14 ... round up 14). In 14 bytes 112 bits of information, of which 14 bits are occupied by the start bit bit 92 bits of payload, 6 unused bits remain (in which you can write some more information, but for simplicity, we will not use them).

Where the 7th bit is a sign of the start bit (indicates the beginning of the packet), 6,5,4,3,2,1,0 are data bits.

The receiving side also knows that it receives a packet of 14 bytes in which the first bit of the first byte is the start bit (1) (in the remaining bytes, the start bits are 0), then 32 bits of the time stamp go in data bits, then 10 ADC measurement bits № 1, then 10 bits of A / D converter number 2, and so on ...

Similarly, the recording on the SD card and reading from it according to the protocol. Total for one day of recording on the SD card we get 115.4 MB of information (14 bytes x 100 measurements per second x 3600 seconds x 24 hours).

Such a data structure is still convenient because later we can select data blocks from any place in the file and display them as graphs, thereby not loading the entire file into RAM (which can reach several tens of gigabytes). And also we will be able to implement convenient scrolling of these graphs by loading the necessary packages.

It's time to start the software implementation for the microcontroller

The library under the microcontroller is written in C ++.

For convenience, create a class:

class PSP { public: /*   init: startBit -   0  1 *arrayByte -      sizeArrayByte -     */ void init(byte startBit, byte* arrayByte, byte sizeArrayByte); /*      pushData: sizeBit -     value -   (       ) */ void pushData(byte sizeBit, uint32_t value); /*       popData: return     . */ byte* popData(); protected: byte startBit; //  byte* arrayByte; //   byte sizeArrayByte; //   byte position = 0; //    bool clearFlag = false; //   void setStartBit(byte &value); //     void clearStartBit(byte &value); //     }; 

With the initialization method, I think everything is clear:

 void PSP::init(byte startBit, byte* arrayByte, byte sizeArrayByte) { this->startBit = startBit; this->arrayByte = arrayByte; this->sizeArrayByte = sizeArrayByte; } 

With the method of adding data is more complicated, here by sly one-by-one manipulations we place the data in our byte array.

 void PSP::pushData(byte sizeBit, uint32_t value) { byte free; byte y; int remBit = 0; //      //   ,     if (!clearFlag) { for (byte i = 0; i < sizeArrayByte; i++) { arrayByte[i] = 0; } clearFlag = true; } //        7      while (remBit > -1) { free = 7 - (position) % 7; //        y = (position) / 7; //     //       remBit = sizeBit - free; //      if (remBit < 0) { arrayByte[y] |= value << ~remBit + 1; //   ,    position += sizeBit; //        remBit = -1; //      } //      else if (remBit > 0) { arrayByte[y] |= value >> remBit; //     ,    position += sizeBit - remBit; sizeBit = remBit; //        } //         else if (remBit == 0) { arrayByte[y] |= value; //    position += sizeBit; remBit = -1; //      } clearStartBit(arrayByte[y]); //   } setStartBit(arrayByte[0]); //   } 

Method for getting an array of packet bytes:

 byte* PSP::popData() { position = 0; //   clearFlag = false; //    return arrayByte; //   } 

And finally, a couple of auxiliary functions:

 //      void PSP::setStartBit(byte &value) { if (startBit == 0) value &= ~(1 << 7); else value |= 1 << 7; } //      void PSP::clearStartBit(byte &value) { if (startBit == 1) value &= ~(1 << 7); else value |= 1 << 7; } 

Let's sum up

As a result of the work done, the “born” compact PSP1N streaming protocol and ready-made libraries that can be downloaded from GitHub can be found here . In this repository you will find:

1. ExampleColsolePSP1N_CSharp / C # library example
2. PSP1N_Arduino / contains a PSP library for working with the protocol and an example of using it on Arduino
3. PSP1N_CSharp / Protocol Library for .NET
4. PSP1N_CSharpTest / unit tests for the library

To demonstrate the operation of the protocol, you can flash a sketch to Arduin and, in the TestingConsole example on the computer, receive data from the microcontroller via the COM port. There, this data is decrypted and displayed in the console application. About the library written in C # for the receiving side, I will tell you another time.

TestingConsole: