On open-source implementations of the GOST R 34.11-2012 hash function and their impact on the electronic signature GOST R 34.10-2012

At one time, the implementation of domestic cryptographic algorithms in the libgcrypt library really inspired me. It became possible to use these algorithms both in Kleopatra and in Kmail and in GnuPg in general, to consider the libgcrypt library as an alternative to openssl with the GOST engine. And everything was great until last Friday.

I was asked to verify the electronic signature GOST R 34.10-2012-256 for a document created in Microsoft Office on MS Windows. And I decided to check it in Kleopatra (I have Linux). And what do you think the signature was wrong. Crept in doubt. I decided to check on openssl with GOST- new engine. The signature has been successfully verified. Immediately re-signed the file in Kleopatra and it did not pass the test on MS Windows. We tried to sign and check other files, everything was fine. There was a question in what a trouble? Since the document hash is involved in signing, it was decided to check the calculation of the hash with different programs. First of all, the open-source implementation for stribog was involved:

• The famous implementation Degtyarev
• Implementation in libgcrypt
• LibreSSL
• Openssl engine

And then there was a shock! The hash counted by the famous implementation of Degtyarev coincided with the hash counted in openssl with GOST endine, but did not coincide with the hash counted using libgcrypt and libressl.

How ru_crypt was right when he wrote at the beginning of his article :

Immediately I warn you that I did not check the correctness of implementations.

By the way, the very standard for GOST R 34.10-2012 also says that test cases are for reference purposes only. It should be clearly understood that test cases do not guarantee that different implementations give the same result for all occasions.
')
To calculate the hash values, the following utilities were used:

1) openssl

$ openssl dgst [–md_gost12_256|-md_gost12_512] <file> 

2) libressl

 $libressl dgst [–streebog256|streebog512] <file> 

3) libgcrypt

 $gchash [stribog256|stribog512] <file> 

4) The famous implementation of Degtyarev

 $gost3411-2012 [-2|-5] <file> 

Here, too, is an interesting thing: in Latin transcription, the stribog write either a stribog or a streebog. It would be nice to come to uniformity. And so it seems that these are different functions. Personally, I prefer the first option - stribog.

Needed an arbitrator.

As an arbitrator, it was decided to use the PKCS # 11 token RUTOKEN EDS-2.0, which supports the Russian cryptographic standards GOST R 34.10-2012, GOST R 34.11-2012, VKO GOST R 34.10-2012 (RFC 7836) with a key length of 256 and 512 bits , and certified by the Federal Security Service of Russia as a means of cryptographic information protection (ICPT) and a means of electronic signature.

In addition, the token RUTOKEN EDS-2.0 is widely distributed and many store certificates for access to government services and other portals.
To calculate the hash value on the token, we use the Tcl language test_digest.tcl script:


When does this discrepancy in implementation occur? So far, we managed to determine that this discrepancy arises when calculating the hash of doc files created in MS Office. And the hash from the first 143 bytes is considered the same, and already when calculating the hash from 144 bytes, the values ​​are different.

The first 143 bytes in hex look like this:

 d0cf11e0a1b11ae1000000000000000000000000000000003e000300feff0900060000000000000000000000010000000100000000000000001000002400000001000000feffffff0000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 

Save them in the file Doc1_143_hex.txt.

The first 144 bytes in hexadecimal look like this:

 d0cf11e0a1b11ae1000000000000000000000000000000003e000300feff0900060000000000000000000000010000000100000000000000001000002400000001000000feffffff0000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 

Save them in the file Doc1_144_hex.txt.

To convert from a hexadecimal form to a binary one, it is convenient to use the hex2bin.tcl script:

 #!/usr/bin/tclsh proc usage {use error} { if {$use == 1} { puts $error puts "Usage:\nhex2bin <file with hex> <file for bin>\n" } } set countcert [llength $argv] if { $countcert != 2 } { usage 1 "Bad usage!" exit } set file [lindex $argv 0] if {![file exists $file]} { usage 1 "File $file not exist" exit } set fd [open $file] chan configure $fd -translation binary set cert_user [read $fd] close $fd if {$cert_user == "" } { usage 1 "Bad file with hex: $file" exit } set cert_user [binary format H* $cert_user] set fd [open [lindex $argv 1] w] chan configure $fd -translation binary puts -nonewline $fd $cert_user close $fd 

Convert a hexadecimal code to binary:

 $./hex2bin Doc1_143_hex.txt Doc1_143.bin $./hex2bin Doc1_144_hex.txt Doc1_144.bin $ 

Now you can check how the hash is calculated by various implementations:
First we consider the hash for the file Doc1_143, bin:

 $ ./openssl dgst -md_gost12_256 Doc1_143.bin md_gost12_256(Doc1_143.bin)= e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 $ ./libressl dgst -streebog256 Doc1_143.bin streebog256(Doc1_143.bin)= e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 $ ./gchash stribog256 Doc1_143.bin e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 Doc1_143.bin $ ./gost3411-2012 -2 Doc1_143.bin GOST R 34.11-2012 (Doc1_143.bin) = e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 $ 

The most important moment has come, the moment of the check on the certified CIPF

 $ ./test_digest.tcl stribog256 Doc1_143.bin Connect the Token and press Enter Loading file for digest: Doc1_143.bin digest_hex= e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 $ 

As you can see, everything ended for good.

Let's see what will happen for the file Doc1_144.bin:

 $ ./openssl dgst -md_gost12_256 Doc1_144.bin md_gost12_256(Doc1_144.bin)= c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 $ ./libressl dgst -streebog256 Doc1_144.bin streebog256(Doc1_144.bin)= 3965c99777eb1b64c783496fe950aa6540bc7baa399a3889995145afbdd76250 $ 

All the hash values ​​do not match. For the purity of the experiment, check the remaining implementations:

 $ ./gchash_1.7.10 stribog256 Doc1_144.bin 3965c99777eb1b64c783496fe950aa6540bc7baa399a3889995145afbdd76250 Doc1_144.bin $ ./gost3411-2012 -2 Doc1_144.bin GOST R 34.11-2012 (Doc1_144.bin) = c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 $ ./test_digest.tcl stribog256 Doc1_144.bin Connect the Token and press Enter Loading file for digest: Doc1_144.bin digest_hex= c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 $ 

The hash calculated by the famous implementation of Degtyarev coincides with the hash calculated in openssl with the GOST engine, but does not coincide with the value of the hash calculated using libgcrypt and libressl.

We will get a similar result if we consider the stribog512 hash.

Conclusion one. If you want the GOST R 34.10-2012 electronic signature, formed by libressl and libgcrypt (and others), to be compatible with the openssl implementation and, most importantly, with the SKZI certified in the certification system of the Federal Security Service of Russia, use proven implementations for calculating hashes. I hope this publication will avoid many misunderstandings, and the authors of the implementation of the stribog in libressl, libgrypt, and possibly others will help to eliminate these discrepancies. Today, it must be admitted, in the above-mentioned products, it is not GOST R 34.10-2012 that is actually implemented, but something else. This is a different algorithm. The given test example, probably, it would be nice to include as a test example for GOST R 34.10-2012. And I am going to edit libgcrypt for Kleopatra and KMail. The tale of Cleopart and Russian cryptography was unfinished.

PS The article was already ready when my colleague said that the divergence of implementations manifest themselves when a sufficiently long sequence of 0xFF occurs. This sequence, by the way, is present at the beginning of the MS Office doc files. I checked it the way it is. The file contained 189 bytes.