New Zopfli algorithm improves zlib compression by 3-8%

In his spare time, one Google employee developed a new Zopfli compression algorithm , which is 3.7–8.3% more efficient than the standard zlib library at the maximum compression level. Initially, the algorithm was created for the lossless compression format of WebP, but it can also be used for other content.

The new algorithm is an implementation of the standard Deflate algorithms, so it is compatible with zlib and gzip, and data unarchiving is already supported by all browsers. It is enough to connect Zopfli on the server. For example, it can be used with the Nginx web server without modification in the gzip module , simply by specifying a new “pre- compressor ”.

True, compression using Zopfli requires about 100 times more resources than gzip, but decompression in the browser is performed at the same speed.

In the article (pdf), the author explains, due to what optimizations, it was possible to achieve an increase in the compression level. Deflate is known to use a combination of a Huffman algorithm and an LZ77 algorithm. The first encodes the characters of the message codes of variable length, depending on the frequency of occurrence of these characters. The second works on the “sliding window” principle, when the second and subsequent occurrences of a certain string of characters in a message are replaced with references to its first occurrence. Existing Deflate implementations use various heuristics to search for suitable occurrences and optimize data analysis before encoding, in order to understand which method is best to use in each case, with building a hash table. The compression level (from -1 to -9) determines the amount of time and resources that is allocated to use heuristics, usually by resizing the strings to search in the hash table.
')
As stated in the author’s article, Zopfli uses more resource-intensive compression techniques based on entropy modeling and the shortest path algorithm in the graph of all possible Deflate representations.

Zopfli allows you to specify compression levels from 5 to 2000. The article (pdf) compares the level of compression in different tests.



On real files, for example, uncompressed jQuery, comparing archivers looks like this :

  File: 268381 bytes
 Zopfli (-i1000) 75,730 bytes, 950 ms
 Gzip (-9) 79388 bytes, 30 ms 

True, in time Zopfli loses to everyone, it works about 81 times slower than the fastest gzip-9 algorithm. Again, it must be emphasized that decompression in the browser is carried out at the same speed.

Installing and using Zopfli:

  git clone https://code.google.com/p/zopfli/
 cd zopfli
 make
 chmod + x zopfli
 ./zopfli> FILENAME>