Comparison of serialization formats

When choosing a serialization format for messages that will be recorded in a queue, a log, or somewhere else, a number of questions often arise that affect the final choice. One of these key issues is the speed of serialization and the size of the received message. Since there are a lot of formats for such purposes, I decided to test some of them and share the results.

Preparation for testing

The following formats will be tested:

1. Java serialization
2. Json
3. Avro
4. Protobuf
5. Thrift (binary, compact)
6. Msgpack

Scala is selected as the PL.
The main tool for testing will be Scalameter .
')
The following parameters will be measured and compared: the time spent on serialization and deserialization, and the size of the resulting files.

Ease of use, the possibility of evolution of the scheme and other important parameters in this comparison will not participate.

Input Generation

For the purity of experiments, it is necessary to pre-generate a data set. The input data format is a CSV file. Data is generated using the simple `Random.next [...]` for numeric values ​​and `UUID.randomUUID ()` for string values. The generated data is written to the csv file using kantan . Total generated 3 sets of data on 100k records each:

1. Mixed data - 28 mb
   
   
   Mixed data

   final case class MixedData( f1: Option[String], f2: Option[Double], f3: Option[Long], f4: Option[Int], f5: Option[String], f6: Option[Double], f7: Option[Long], f8: Option[Int], f9: Option[Int], f10: Option[Long], f11: Option[Float], f12: Option[Double], f13: Option[String], f14: Option[String], f15: Option[Long], f16: Option[Int], f17: Option[Int], f18: Option[String], f19: Option[String], f20: Option[String], ) extends Data 

   
   
2. Only lines - 71 mb
   
   
   OnlyStrings

    final case class OnlyStrings( f1: Option[String], f2: Option[String], f3: Option[String], f4: Option[String], f5: Option[String], f6: Option[String], f7: Option[String], f8: Option[String], f9: Option[String], f10: Option[String], f11: Option[String], f12: Option[String], f13: Option[String], f14: Option[String], f15: Option[String], f16: Option[String], f17: Option[String], f18: Option[String], f19: Option[String], f20: Option[String], ) extends Data 

   
   
3. Only numbers (long) - 20 mb
   
   
   Onlylongs

    final case class OnlyLongs( f1: Option[Long], f2: Option[Long], f3: Option[Long], f4: Option[Long], f5: Option[Long], f6: Option[Long], f7: Option[Long], f8: Option[Long], f9: Option[Long], f10: Option[Long], f11: Option[Long], f12: Option[Long], f13: Option[Long], f14: Option[Long], f15: Option[Long], f16: Option[Long], f17: Option[Long], f18: Option[Long], f19: Option[Long], f20: Option[Long], ) extends Data 

   
   

Each entry consists of 20 fields. The value of each field is optional.

Testing

The characteristics of the PC on which the testing took place, the version of scala and java:
PC: 1.8 GHz Intel Core i5-5350U (2 physical cores), 8 GB 1600 MHz DDR3, SSD SM0128G
Java version: 1.8.0_144-b01; Hotspot: build 25.144-b01
Scala version: 2.12.8

Java serialization

Json

Avro

Avro circuit generated on the fly before direct testing. For this library avro4s was used .

Protobuf

To generate protobuf3 classes, the ScalaPB plugin was used .

Thrift

To generate scala-like thrift classes, the Scrooge plugin was used.

Msgpack

Final comparison

Given the fact that the results are not completely accurate, based on them, you can still make some observations:

1. Once again convinced that java serialization is slow and not the most economical in terms of the volume of output data. One of the main reasons for slow work is to refer to the fields of objects with the help of reflection. By the way, the access to the fields and their further recording does not occur in the order in which you declared them in the class, but in sorted in lexicographical order. This is just an interesting fact;
2. Json is the only text format presented in this comparison. Why data serialized in json takes up a lot of space - each record is recorded with the schema. It also affects the speed of writing to the file: the more bytes you need to write, the longer it takes. Also, do not forget that a json-object is created for each record, which also does not reduce the time;
3. When serializing an object, Avro analyzes the schema in order to decide how to handle a particular field. These are additional costs leading to an increase in total serialization time;
4. Thrift, compared to, for example, protobuf and msgpack, requires more memory to write a single field, since its meta information is saved along with the field value. Also, if you look at the output files of thrift, you can see that various identifiers of the beginning and end of the record and the size of the entire record as a separator occupy a small fraction of the total. All this certainly only increases the time spent on packaging;
5. Protobuf, like thrift, packs meta information, but makes it somewhat more optimized. Also, the difference in the packing and unpacking algorithm itself allows this format to work faster than others in some cases;
6. Msgpack works pretty fast. One of the reasons for the speed is the fact that no additional meta information is serialized. This is both good and bad: good because it takes up little space on the disk and does not require additional recording time, bad because in general nothing is known about the recording structure, therefore the definition of how to package and unpack something or a different value is performed for each field of each record.

As for the size of the output files, the observations are quite unambiguous:

1. The smallest file for a numeric set turned out in msgpack;
2. The smallest file for string typing was found in the source file :) Apart from the source file, avro won by a small margin from msgpack and protobuf;
3. The smallest file for mixed-set again turned out at msgpack. However, the gap is not so noticeable and very close are avro and protobuf;
4. The biggest files turned out at json. However, it is necessary to make an important note - the json text format and comparing it with binary ones in terms of volume (and also in terms of serialization speed) is not entirely correct;
5. The largest file for a numeric set turned out in standard java serialization;
6. The largest file for a string set turned out at thrift binary;
7. The largest file for the mixed set turned out at thrift binary. Behind him comes the standard java serialization.

Format analysis

Now let's try to understand the results obtained using the example of serializing a string with a length of 36 characters (UUID) without taking into account separators between records, different identifiers of the beginning and end of the record - only a record of 1 string field, but taking into account such parameters as, for example, the type and number of the field . Consideration of serialization of a line quite covers several aspects at once:

1. Number serialization (in this case, the string length)
2. String serialization

Let's start with avro. Since all fields are of type `Option`, the scheme for such fields will be the following:` union: [“null”, “string”] `. Knowing this, you can get the following result:
1 byte to specify the type of record (null or string), 1 byte for the length of the string (1 byte because avro uses variable-length to write integers) and 36 bytes for the string itself. Total: 38 bytes.

Now consider the msgpack. Msgpack uses integer-like approach to write integers: spec . Let's try to calculate how much it actually takes to write a string field: 2 bytes for the length of the string (since the string is> 31 bytes, then 2 bytes are needed), 36 bytes for the data. Total: 38 bytes.

Protobuf also uses variable-length to encode numbers. However, in addition to the length of the string, protobuf adds another byte with the number and type of the field. Total: 38 bytes.

Thrift binary does not use any optimization for writing the length of a string, but instead of 1 byte for the number and type of the field, thrift takes 3. Therefore, the following result is obtained: 1 byte for the field number, 2 bytes for the type, 4 bytes for the length of the string, 36 bytes for string. Total: 43 bytes.

Thrift compact , unlike binary, uses the variable-length approach to record integers and additionally use the abbreviated header of the field if possible. On this basis, we get: 1 byte for the type and field number, 1 byte for the length, 36 bytes for the data. Total: 38 bytes.

Java serialization took 45 bytes to write a string, of which 36 bytes is a string, 9 bytes is 2 bytes per length, and 7 bytes for some additional information, which I could not decrypt.

Only avro, msgpack, protobuf and thrift compact are left. Each of these formats will require 38 bytes to write utf-8 strings with a length of 36 characters. Why, then, with the packing of 100k string records, a smaller volume turned out to be in avro, although not a compressed scheme was also recorded with the data? Avro has a small gap from other formats and the reason for this gap is the absence of additional 4 bytes per package of the length of the entire record. The fact is that neither msgpack, nor protobuf, nor thrift have a special record separator. Therefore, in order for me to correctly unpack the records back, I needed to know the exact size of each record. If it were not for this fact, then, with a high probability, the smaller file would be in msgpack.

For a numeric data set, the main reason for the msgpack win was the lack of schema information in the packed data and the fact that the data was sparse. In thrift and protobuf, even empty values ​​will take more than 1 byte due to the need to pack information about the type and number of the field. Avro and msgpack require exactly 1 byte to write an empty value, but avro, as already mentioned, saves the scheme with the data.

Msgpack also packed a smaller file and a mixed set, which was also sparse. The reasons for this are all the same factors.

Thus, it turns out that the data packed in msgpack occupies the least space. This is quite a valid statement - it’s not for nothing that msgpack was chosen as the storage format for tarantool and aerospike.

Conclusion

After testing, I can draw the following conclusions:

1. Getting stable benchmark results is difficult;
2. Format selection is a trade-off between serialization speed and output size. At the same time, one should not forget about such important parameters as the usability of the format and the possibility of schema evolution (often these parameters play a dominant role).

Source code can be viewed here: github