Zoo communication protocols for GPS trackers (part 2)

This is a continuation of the article about network protocols used with GPS trackers. If in the first part we discussed frame decoders, then in the second part we will look at the options for coding various types of payloads. Many manufacturers of devices for GPS monitoring develop their own application-level protocols for transferring data from the device to the server, and sometimes developers resort to various sophisticated and not always understandable solutions for data coding.

For those who have not read or forgotten the first part, I remind you that there are two main groups into which all protocols can be divided: text and binary. In text protocols, information is transmitted in the form of ASCII codes, and in binary, respectively, all data is encoded in big-endian or, more rarely, in little-endian binary form.

Byte order (endianness)

The order from high to low (big-endian) is standard for TCP and UDP protocols, it is used in data packet headers and in many higher level protocols. Therefore, the byte order from high to low is often called network byte order (network byte order), and it is the de facto standard for coding binary data in network protocols.
')

  short i = 1;  // 0x00 0x01 - big-endian (network byte order)
              // 0x01 0x00 - little-endian 

The problem is that CPUs most often use the order from low to high (little-endian), and therefore many manufacturers of GPS trackers decided to encode data in the same format that the data is stored in memory (ie, little-endian) . The list of such manufacturers is quite large, so I will give only a few examples: Baltic Car Equipment (BCE, Lithuanian company), Cellocator (a large international company with headquarters in Israel) and GalileoSki (a well-known Russian manufacturer of GPS trackers with headquarters in Perm).

Some manufacturers go even further and encode part of the data with one byte order, and some with another. For example, the devices of the Chinese company Topflytech and the Polish manufacturer Tytan GPS send all data in standard network byte order, except for floating-point numbers, which are sent in little-endian format. In fairness it should be noted that Tytan GPS fixed the problem in the second version of its protocol.

Geographical coordinates

The coordinates (latitude from -90 ° to + 90 °, longitude from -180 ° to + 180 °) can be represented in various ways. GPS receivers usually return a value in degrees and minutes with a decimal fraction, with the positive coordinates represented by the letters "N" (north latitude) and "E" (east longitude), and the negative coordinates - the letters "S" (south latitude) and " W ”(west longitude).

The greatest variety of options for coding coordinates can be found in text protocols. The sign of the coordinates can be either a letter or a sign in front of the value in degrees (in some cases, a positive sign is clearly indicated). The value can be represented either in degrees with decimal fractions or in integer degrees and minutes by decimal. Decimal points may be omitted, and there may also be no separator between latitude and longitude. Here, for example, several options for the representation of the Pentagon coordinates:

  38.870989, -77.055961 // degrees by decimal
 3852.25934, N, 07703.35766, W // degrees (2 signs for latitude and 3 signs for longitude) and minutes by decimal
 W77.055961, N38.870989 // hemisphere and degrees by decimal
 + 38.870989, -77.055961 // explicit indication of north latitude plus sign
 3852.2593N07703.3577E // no separator between coordinates 

In binary protocols, geographic coordinates are usually represented either by an integer that needs to be divided by a certain factor to get coordinates, or by a floating-point number (usually 4 bytes).

  0x04 0x2b 0x9f 0xa4 -> 69967780 -> (/ 60/30000) -> 38.870989 // integer
 0xc2 0x9a 0x1c 0xa7 -> -77.055961 // floating point number (float) 

The coordinate sign can be stored separately from the coordinate value. For example, in a protocol from Tzone Digital Technology, the bits of the latitude and longitude are combined with the direction value in 2 data bytes:

  0x03 0x0E // 0 0 0 0 0 0 1 1 0 0 0 0 1 1 1 0
           // |  |  |  Direction (9 bits) - 270 degrees
           // |  |  longitude sign - minus (west longitude)
           // |  sign of latitude - plus (north latitude) 

Another interesting variant of coding coordinates in binary is a binary-decimal code (binary-coded decimal or simply BCD). The essence of coding is that each decimal digit of a number is written in the form of its four-bit binary code. For example, the latitude of the location of the Pentagon in the protocols of Chinese companies KHD or Gator will look as follows:

  0x03, 0x85, 0x22, 0x59, 0x34 -> 38 degrees 52.25934 minutes 

Please note that the entry in hexadecimal is exactly the same as the value in decimal. It turns out that the data encoded in BCD is relatively easy to read by humans, but unfortunately, this type of encoding makes it harder for the computer to read this data, and also increases the size of the message.

date and time

The overwhelming majority or even all protocols, when transmitting coordinates to the server, contain one or several time stamps. Usually, this is the time for determining the coordinates of the GPS receiver, but sometimes the date and time of the internal clock of the GPS tracker at the time when the package was formed. Some trackers send two or even more timestamps.

In text protocols, almost always time and date are divided into separate components. Here are some examples of the presentation of the time of the last clock transfer in Russia (October 26, 2014, 2 am):

  2014/10 / 26,02: 00: 00 // date and time with delimiters
 020000.000 ... 261014 // time accurate to milliseconds
 141026 ... 020000 // date and time without separators
 20141026020000
 10/14/26 2:00 // date and time, accurate to minutes 

An important point that is worth noting is that usually time is transferred around GMT (UTC or GMT time zones). This is important because the server and the tracker can be in different time zones. Some GPS trackers allow you to configure the time zone. In this case, some of them report the selected time zone to the server, and some - not. For those that do not report, you must manually set the zone on the server.

One interesting example of a problem with time zones is a nameless protocol used in many cheap Chinese-made trackers. The tracker sends the local date (year, month, day) and time (hours, minutes, seconds), as well as the time with an accuracy of milliseconds GMT. Interesting in this case is that having this data, you can calculate the GMT date for the time zones from GMT-12 to GMT + 12. For example, if at the entrance we have 2016-01-01 01:00:00 (local time) and 59: 00: 00.000 (GMT), then the result at the output will be 2015-12-31 59: 00: 00.000 (GMT). Unfortunately, this method does not work for time zones with a deviation from Greenwich more than 12 hours.

Another remarkable time representation is used in the standard TAIP protocol. The time is presented in the form of weeks, days and seconds from January 6, 1980. The significance of this date is not entirely clear, but it is standard for many GPS receivers.

Some protocols transfer to the server only the current time of day. In this case, the only thing that can be done is to substitute the current date for the server version. Accordingly, this can lead to problems if the server and device time is different or any delay in data transfer. For example, a device can transmit a time of 11:59:59 PM, and upon receiving this message, the server will already have the beginning of the next day. One way around this problem is to compare the received timestamp with the current server time, and if the difference is more than an hour or several hours, then you need to add or subtract one day from the received date.

In binary protocols, the date and time are either split into separate components in the same way as in text protocols, or they are stored as a UNIX timestamp (number of seconds or milliseconds since midnight (00:00:00 GMT) on January 1, 1970).

Conclusion

All the information presented in this article was accumulated in the course of work on the GPS monitoring server. The project is completely Open Source, and if anyone is interested, then the source code with the implementation of all the above protocols can be found on GitHub .

Also, someone may be interested in the documentation on the protocols for GPS-trackers . Unfortunately, I cannot publish all the available documents in the public domain, as some of them are confidential.