You are dangerously incompetent in cryptography
From the translator: Although the promise of the article Najaf Ali , translated below, is slightly advertising (“leave the cryptography to us, the experts”), but the examples described in it seemed to me quite interesting and worthy of attention.
In addition, it will never be superfluous to repeat a simple truth: do not invent your crypto protection. And this article perfectly illustrates why.
There are four stages of competence:
We all start with the first stage, whether we like it or not. The key to moving from stage 1 to stage 2 in any area is to make a lot of mistakes and get feedback. If you get a response, you begin to understand what you did right, what went wrong, and what you should improve next time.
Cryptography is dangerous because you do not get a response when you do something wrong. For the average developer, one block of random bytes encoded in base 64 is as good as any other.
')
You can learn to program well unintentionally. If your code does not compile, does not do what you need of it, or contains easily detectable bugs, you get an immediate response. You fix everything, and next time you’ll get better.
Cryptography is impossible to learn unintentionally. If you don’t spend time reading materials about vulnerabilities and trying to use them, your home-grown cryptography-based defense mechanisms don't have much chance against real attacks.
If you don’t pay a security expert who knows how to break open cryptographic security mechanisms, you won’t be able to know that your code is insecure. Attackers who bypass your defense will not help you either (ideally, they will be able to bypass it so that you never know about it).
Below are a few examples of the misuse of cryptography. Consider, if you did not read this post, could you detect these errors in reality?
One photo-sharing site somehow authenticated requests to its API as follows:
To make sure that the request actually came from the specified user, the server similarly generates a signature for the request parameters and the secret that is written to him for this user.
The code for this could be:
With basic knowledge of how MD5 works, this is a completely adequate implementation of authenticating API requests. Looks quite safe, right? Are you sure?
It turns out that such a scheme is vulnerable to the so-called " length extension attack ".
In short:
Any developer who did not know about this in advance would have easily fallen. The developers Flickr, Vimeo and Remember the Milk used this approach in their products (pdf).
The point is not that you need to know every hidden detail about the internals of cryptographic functions. The bottom line is that there are a million ways to make mistakes with cryptography . So do not touch it.
Not convinced? Well, let's try to fix this example and see if we can make it safe ...
Your familiar hacker told you about this vulnerability and recommended using the Hash-based Message Authentication Code ( HMAC ) to authenticate requests.
Fine! HMAC coined just for our case. And the replacement is very simple, almost one-to-one.
Our signature verification code on the server can now look like this:
Looks quite safe, right? Are you sure?
It turns out that the code above is vulnerable to a time attack ( Timing attack ), which allows you to choose the right HMAC for a given message.
In short:
Using the technique described above, you can reliably determine the HMAC for any request with which you would like to call the API, and successfully pass the authentication.
Again, it is possible that you did not know about the attacks in time, but you are not expected to do this. The point is not that you should know the details of specific vulnerabilities and beware of them. The bottom line is that there are a million ways to make mistakes with cryptography . So do not touch it.
And again, let's continue and try to make this code more secure ...
You can avoid time attacks by comparing the sent and calculated HMAC in a time-insensitive way. Those. you cannot rely on the built-in string comparison operator in your programming language, since it will immediately exit as soon as it finds the first non-matching character.
To compare strings, we can use the fact that XOR of any byte with itself will give 0. All we need to do is apply the XOR operation to each pair of corresponding bytes from rows A and B, add the resulting results and return true if the sum is equal to 0, and false otherwise.
On Ruby, this might look something like this:
Looks quite safe, right? Are you sure?
I doubt. This is already the limit of my knowledge of the potential vectors of attacks on schemes of this type. But I'm not sure that there is no way to hack this either.
Get rid of problems. Do not use cryptography. This is plutonium. There are millions of ways to make mistakes and just a few valuable ways to get it right.
PS If you can’t do without manually checking the HMAC and you have access to the activesupport module, you can perform time-insensitive comparison using
PPS Still not convinced? Complete the Matasano Crypto Challenges - maybe at least they change your mind. I haven’t passed half yet, and I already had to contact two past clients to fix their cryptography errors.
About the author: Najaf Ali is a technical consultant from London. Engaged in software development for startups and small and medium businesses. He writes about technology, entrepreneurship and software development.
In addition, it will never be superfluous to repeat a simple truth: do not invent your crypto protection. And this article perfectly illustrates why.
There are four stages of competence:
- Unaware incompetence - when you do not know that you are incompetent and how extensive your incompetence is.
- Aware incompetence - when you know about your incompetence and know what steps you need to take to improve the situation.
- Conscious competence - when you are good and you know about it. (This is cool!)
- Unrecognized competence - when you are so good that you no longer know about it.
We all start with the first stage, whether we like it or not. The key to moving from stage 1 to stage 2 in any area is to make a lot of mistakes and get feedback. If you get a response, you begin to understand what you did right, what went wrong, and what you should improve next time.
Cryptography is dangerous because you do not get a response when you do something wrong. For the average developer, one block of random bytes encoded in base 64 is as good as any other.
')
You can learn to program well unintentionally. If your code does not compile, does not do what you need of it, or contains easily detectable bugs, you get an immediate response. You fix everything, and next time you’ll get better.
Cryptography is impossible to learn unintentionally. If you don’t spend time reading materials about vulnerabilities and trying to use them, your home-grown cryptography-based defense mechanisms don't have much chance against real attacks.
If you don’t pay a security expert who knows how to break open cryptographic security mechanisms, you won’t be able to know that your code is insecure. Attackers who bypass your defense will not help you either (ideally, they will be able to bypass it so that you never know about it).
Below are a few examples of the misuse of cryptography. Consider, if you did not read this post, could you detect these errors in reality?
API Authentication for Your Photo Sharing Site
Message Authentication with MD5 + Secret
One photo-sharing site somehow authenticated requests to its API as follows:
- The user has the following details:
- public user id by which they identify themselves (it is safe to send it in plain text)
- shared secret with the server with which they sign messages (must be kept secret)
- The user makes a request to the API via HTTP (or HTTPS is not important). Destructive changes are performed using a POST / GET request with special parameters (for example,
{ action: create, name: 'my-new-photo' }). - To authenticate the message, the user sends his user id as a parameter and signs the message with his private key. The signature is MD5 from a string consisting of a shared secret followed by key-value pairs.
To make sure that the request actually came from the specified user, the server similarly generates a signature for the request parameters and the secret that is written to him for this user.
The code for this could be:
# require 'openssl' ## user_id = '42' secret = 'OKniSLvKZFkOhlo16RoTDg0D2v1QSBQvGll1hHflMeO77nWesPW+YiwUBy5a' ## , params = { foo: 'bar', bar: 'baz', user_id: user_id } ## MAC message = params.each.map { |key, value| "#{key}:#{value}" }.join('&') params[:mac] = OpenSSL::Digest::MD5.hexdigest(secret + message) ## - ... HTTP.post 'api.example.com/v3', params # ## user = User.find(params[:user_id]) secret = user.secret ## MAC challenge_mac = params.delete(:mac) ## MAC , , message = params.each.map { |key, value| "#{key}:#{value}" }.join('&') calculated_mac = OpenSSL::Digest::MD5.hexdigest(secret + message) ## MAC if challenge_mac == calculated_mac # - , else # , end With basic knowledge of how MD5 works, this is a completely adequate implementation of authenticating API requests. Looks quite safe, right? Are you sure?
It turns out that such a scheme is vulnerable to the so-called " length extension attack ".
In short:
- If you know the value of
md5('foo'), by the method of calculating MD5, you can very easily calculate the value ofmd5('foobar')without even knowing the prefix 'foo'. - So if you know the value of
md5('secretfoo:bar'), you can easily calculate the value ofmd5('secretfoo:bar&bar:baz')without even knowing the prefix 'secret'. - This means that if you have at least one signed message, you can forge the signatures for this message with any additional parameters in the request. And authentication check according to the above scheme will be successful.
Any developer who did not know about this in advance would have easily fallen. The developers Flickr, Vimeo and Remember the Milk used this approach in their products (pdf).
The point is not that you need to know every hidden detail about the internals of cryptographic functions. The bottom line is that there are a million ways to make mistakes with cryptography . So do not touch it.
Not convinced? Well, let's try to fix this example and see if we can make it safe ...
Message Authentication with HMAC
Your familiar hacker told you about this vulnerability and recommended using the Hash-based Message Authentication Code ( HMAC ) to authenticate requests.
Fine! HMAC coined just for our case. And the replacement is very simple, almost one-to-one.
Our signature verification code on the server can now look like this:
require 'openssl' ## user = User.find(params[:user_id]) secret = user.secret ## HMAC challenge_hmac = params.delete(:hmac) ## HMAC ## , message = params.each.map { |key, value| "#{key}:#{value}" }.join('&') calculated_hmac = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('md5'), secret, message) ## HMAC if challenge_hmac == calculated_hmac # - , else # , end Looks quite safe, right? Are you sure?
It turns out that the code above is vulnerable to a time attack ( Timing attack ), which allows you to choose the right HMAC for a given message.
In short:
- For a given message, try sending it with an HMAC consisting of repeating the same character multiple times. Repeat this for each ASCII character - 'aaaa ...', 'bbbb ...', etc.
- Measure the processing time of each request. Since string comparison works a little longer, if the first characters match, the message that will be processed longer than the others will contain the correct first HMAC character.
- Noise from delays can be smoothed in two ways:
- Perform each query a couple of hundreds or even thousands of times and use the average time.
- Run your attack code in the same data center where the attacked application is located. If it’s impossible to determine a data center, in the worst case, you can rent a server from each major provider and find out which of them will have a significantly smaller ping for the target.
- When you have calculated the first character, repeat the queries with matching characters from the second and further. For example, if the matched first character is 'x', send requests with HMAC 'xaaa ...', 'xbbb ...', etc.
- Continue until you get the entire HMAC.
Using the technique described above, you can reliably determine the HMAC for any request with which you would like to call the API, and successfully pass the authentication.
Again, it is possible that you did not know about the attacks in time, but you are not expected to do this. The point is not that you should know the details of specific vulnerabilities and beware of them. The bottom line is that there are a million ways to make mistakes with cryptography . So do not touch it.
And again, let's continue and try to make this code more secure ...
Check HMAC in a timeless manner
You can avoid time attacks by comparing the sent and calculated HMAC in a time-insensitive way. Those. you cannot rely on the built-in string comparison operator in your programming language, since it will immediately exit as soon as it finds the first non-matching character.
To compare strings, we can use the fact that XOR of any byte with itself will give 0. All we need to do is apply the XOR operation to each pair of corresponding bytes from rows A and B, add the resulting results and return true if the sum is equal to 0, and false otherwise.
On Ruby, this might look something like this:
require 'openssl' ## , def secure_equals?(a, b) return false if a.length != b.length a.bytes.zip(b.bytes).inject(0) { |sum, (a, b)| sum |= a ^ b } == 0 end ## user = User.find(params[:user_id]) secret = user.secret ## HMAC challenge_hmac = params.delete(:hmac) ## HMAC ## , message = params.each.map { |key, value| "#{key}:#{value}" }.join('&') calculated_hmac = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('md5'), secret, message) ## HMAC if secure_equals?(challenge_hmac, calculated_hmac) # - , else # , end Looks quite safe, right? Are you sure?
I doubt. This is already the limit of my knowledge of the potential vectors of attacks on schemes of this type. But I'm not sure that there is no way to hack this either.
Get rid of problems. Do not use cryptography. This is plutonium. There are millions of ways to make mistakes and just a few valuable ways to get it right.
PS If you can’t do without manually checking the HMAC and you have access to the activesupport module, you can perform time-insensitive comparison using
ActiveSupport::MessageVerifier . Do not write it from scratch. And for God's sake, do not copy my implementation above .PPS Still not convinced? Complete the Matasano Crypto Challenges - maybe at least they change your mind. I haven’t passed half yet, and I already had to contact two past clients to fix their cryptography errors.
About the author: Najaf Ali is a technical consultant from London. Engaged in software development for startups and small and medium businesses. He writes about technology, entrepreneurship and software development.
Source: https://habr.com/ru/post/181372/
All Articles