πŸ“œ ⬆️ ⬇️

Blowfish on guard ivi

There are often contradictions between developers and products. The first ones are closer to the hardware and are responsible for it, the second ones are for the user interface. Backend-developers do not want to load the system once again, store deliberately false unverified data, waste processor time and disk space. In their worldview, a priori, all requests, all users are intruders who just want to score all the memory and disable the system. If not for vandalism, then for the sake of competitors of similar services.



For example, there are no problems to take and register a user in the database using the β€œemail and password” combination. If we do not send any mailings, do not work with monetary systems and user identification by his email is required solely for his convenience, then we still have a question about the hardware resources of our systems. By running an infinite loop with requests for registration, we can drive all the disks, and the system will fail. Therefore, we have to force users to make such inconvenient steps in the form of confirming their email. The same thing happens with reference to phone numbers. The same is true for linking social network accounts that have been delegated user identification tasks.

There will always be those who are able to raise their own mail server. Creating virtual mailboxes, clicking on confirmation links in them is trivially automated, which again can damage server resources. However, in reality this happens so rarely that against the general statistical background of all requests it will be clearly traceable and the consequences are preventable. A rarity: due to the high threshold of entry for registration in the system. It is the entry threshold that, in its majority, eliminates the attempts of intruders.
')

The threshold of entry of mere mortals


The ideal world of product managers does not contain any complex technical systems in the form of postal systems, referrals and other things. The ideal end result for them is when the user, or rather his software client (a program in the browser, among other things), identifies and authenticates himself as a unique entity. The user connects to the service, and he immediately recognizes it without unnecessary gestures in the form of SMS-ok and links. In addition to unnecessary actions, this requires the user and binding to third-party services, for which he, too, will have to spend time.

The ideal solution to the forehead is when the program simply generates a random large number (random enough so that it does not coincide with anyone else) and will use it as an identifier. Each user is his own service and authentication tool. We assume that it has a reliable secure long-term storage of this identifier and all communication with us occurs over an encrypted trusted channel ( TLS ).

There remains one β€œbut”: the threshold for entering such software solutions is almost zero for registration. A software client can be both a good person and forgotten about the laws of Azimov as a robot.

As a solution, you can enter something that would distinguish a robot from a human: CAPTCHA . Quite simply implemented and working effectively. At a minimum, it gives a sufficient entry threshold to weed out many vandals (but not competitors who buy Chinese discriminator farms). However, such a decision is strongly disliked both by users breaking eyes and nerves, and, accordingly, by products.

Prove that worked


One of the solutions we use is a good old proof-of-work (PoW) system, known since the beginning of email spam, which (whether it is a robot or not) did not have enough data to identify. Once in our context, we want to secure the waste of our resources, since sending data by the client is a much cheaper operation, then PoW is a means of restoring equity because the user will have to work out their right to process the request.

Proof-of-work is a technology in which it is necessary to expend the minimum established work in order to perform an action. This work is resource-intensive, but verification of the success of its implementation is cheap. If someone wants to spend our resources, then with a hundredfold difference between the check and the solution to the work, the attacker will have to spend a hundred times more resources than necessary. As a rule, it will be economically disadvantageous.

The user does not have to enter any CAPTCHA or depend on third-party services. It only has to warm up its processor a bit, transparently for the interface.

The client-server operation scheme is simple:

 ──────┐ β”Œβ”€β”€β”€β”€β”€β”€β”
 β”‚Customerβ”‚ β”‚Serverβ”‚
 β”€β”€β”¬β”€β”€β”€β”˜ β””β”€β”€β”¬β”€β”€β”€β”˜
    β”‚ JSON-RPC REQUEST β”‚
    ─────────────────────────────>
    β”‚ β”‚
    β”‚ TASK β”‚
    β”‚ <─────────────────────────────
    β”‚ β”‚
    ────┐ β”‚
    β”‚ β”‚ Solve (TASK)
    β”‚ <β”€β”€β”€β”˜ β”‚
    β”‚ β”‚
    β”‚ REQUEST, TASK, SOLUTION β”‚
    ─────────────────────────────>
    β”‚ β”‚
    │────┐
    β”‚ β”‚ β”‚ Check (TASK, SOLUTION)
    β”‚ β”‚ <β”€β”€β”€β”˜
    β”‚ β”‚
    β”‚ Result of the request
    β”‚ <─────────────────────────────

The data bound to the task conditions are not stored in the database and therefore must be authenticated, for example, using cryptographic signatures . And in order to avoid the possibility of repeating the request, then at least, for example, tie to the time and give clear deadlines for the task.

If the client's request is a JSON-structure REQ , then the client in the response from the server receives:

{"req": REQ, "bestbefore": 1234, "task": TASK, "sign": "SIGN"} 

The client sends in response:

 {"req": REQ, "bestbefore": 1234, "task": TASK, "sign": "SIGN", "answer": RESULT} 

where bestbefore is the time until which a given query can be executed, TASK is the initial conditions of the problem, and SIGN is the cryptographic signature of the entire serialized dictionary. RESULT is a solution to a problem.

If the client tries to change his original request, the signature will be invalid. If he tries to do it again, after some time (for example, after the comment is published), the server, having checked with his watch, invalidates the request. If the client tries to substitute a previously solved task, the signature will also be invalid.

Instead of time binding, you can issue one-time tokens that are randomly generated. For the sake of simplicity, you can keep it in Redis and make a limited lifetime. Upon request, we read 16 bytes (128 bits) from / dev / urandom and put this line in Redis for ten seconds. This token should appear in the client's response. If it is still in Redis, then we delete and process the request. When reused, the server no longer knows such a token. Their short time of life guarantees us that it will not be possible to score and disable Redis, except that only with large communication channels and capacities.

It would be even better to add some unique identifier to the request, leaving the bestbefore . If the task is successfully verified, we will save this identifier in Redis with the remaining lifetime. Thus, when issuing tasks, we do not save anything. The user will have to solve the problem to try to repeat the action, but only within the remaining time he has the opportunity to commit them, while it is really bestbefore . For a server, this translates into a cheap PoW check and a key existence check in Redis.

 TASK_SIGN_KEY = "ff7555ded9526ecf03b1617a61514d30".decode("hex") TASK_LIFETIME = 60 def task_generate(req): result = { "req": req, "bestbefore": datetime.utcnow() + timedelta(seconds=TASK_LIFETIME), "task": task_new(), "id": urandom(16).encode("hex"), } result["sign"] = sign(TASK_SIGN_KEY, serialize(result)) return result def task_validate(req): if req["bestbefore"] < datetime.utcnow(): return False signature = req.pop("sign") answer = req.pop("answer") if not constant_time_compare(signature, sign(TASK_SIGN_KEY, req)): return False if redis.exists(req["id"]): return False redis.setex(req["id"], TASK_LIFETIME) if not task_verify(req["task"], answer): return False return True 

Do not forget that any strings used in cryptography and requiring comparison should not be compared simply out of the box with native language tools. Comparison of "==" will be optimal in performance and, roughly speaking, the operation will be completed at the first unmatched byte: as a result, the test time is not constant and because of this you can make powerful attacks in the case of TLS that are able to decrypt the entire message. Frequent practice (though not optimal in terms of performance) is a comparison through HMACs:

 hmac(compare_key, string1) == hmac(compare_key, string2) 

At the same time, the comparison time is not constant, but randomized, which will prevent attacks aimed at leaking string comparison time information.

Tasks


What can these tasks represent? At the first approach to the projectile, we decided to use a part already from the 1970s of the famous Merkle puzzles . Officially, this can be considered the first asymmetric key exchange system, looking at which the Diffie-Hellman algorithm was invented. Each puzzle is an encrypted line and a part of the decryption key. If this is the DES encryption algorithm, then a full brute-force search takes an average of 2 ^ 55 operations. We can give 5 bytes (40 bits) of this key, and then it will be necessary to do 2 ^ 15 operations, that is, more than 32 thousand decryption operations. The result of the decision will be the rest of the key: the missing two bytes. Checking the solution on the server is a single decryption operation. Previously it is possible to agree that the line β€œYOUBROKE” is encrypted.

It is not very difficult to estimate what the parameters of such tasks should be. We take the lowest-performing device supported, which can be found in large quantities among users. We assume that there is old software with not very optimal implementations of algorithms (for example, it is JavaScript in the browser). We set the maximum timeout for the user and see how many iterations can be performed. We understand how many bits of the key not to disclose.

What encryption algorithm do you prefer? The choice is huge. It would be desirable to be legally clean when using it, not to pay, so that it is both fast and cryptographically secure (although we are not secrets to transmit). Most importantly, at the same time its implementations are on all platforms. Among the platforms supported by ivi are Web browsers, iOS, Android, SmartTV and others. Everywhere its developers, who, of course, would not want to write a lot of code. JavaScript does not have any built-in cryptographic tools. If AES library implementation was everywhere, then the choice would be obvious - the path of least resistance (if you don’t get involved with the state, then GOST is not dictated).

The choice fell on Blowfish - the block cipher known since 1993, developed by the famous cryptographer Bruce Schneier . It fully satisfies the set conditions, has completely free and open implementations. Yes, there are certainly more technically advanced algorithms, but their code is either more complicated, or the algorithm is not so well known, or there are not so many implementations in various languages. Ciphers such as Arcfour and Salsa20 are simpler, but they are streamed, and this limits the scope of application, which will be discussed below. In general, the XTEA cipher is also excellent, but human subjectivity is everywhere present and Schneier's weight was decisive.

In addition, in all the tasks below, this cipher is used only in encryption mode, without requiring code to decrypt. Since this is a Feistel network , decryption is simply an inversion of the course of operations, but it also takes its share of unused code.

Birthdays


The task of finding the decryption key has a serious drawback: there is too much difference in performance between user devices. The JavaScript code in the browser on the ARM processor is several orders of magnitude slower than what will be running on a modern PC with implementation on C. The quantitative difference becomes qualitative: either the user of mobile devices waits too long for a PoW solution, or the complexity on the PC becomes insignificant and sending traffic It will be much more expensive.

The available resource available on all platforms except the processor is RAM. The tasks of the processors can be implemented in hardware in the form of ASIC and FPGA solutions with higher performance per currency unit. But the difference in memory performance on various platforms available to users is much less significant.

A well-known algorithm for storing passwords scrypt uses random access memory to speed up calculations. If everything is done on the processor, the calculations will take a very long time. The difference is so great that ASIC / FPGA or powerful Intel Xeon processors will play absolutely no role if there is no memory.

The solution adopted by ivi is the use of memory as a cache in the problem of searching for collisions between the results of the PRP function (pseudo random permutation is a pseudo-random permutation).

In probability theory there is a paradox of birthdays : if a group of 23 or more people is given, then the probability that at least two of them have birthdays (number and month) coincide, exceeds 50%. If we take a hash function with a 128-bit output, then the probability that two different inputs match the hash value will be one per 2 ^ 64.

As a certain hash value, we use part of an encrypted piece of data. For example, we take only three bytes from the encrypted data (we have 24-bit similarity to the hash function). The server sends the key to be used for encryption. In the client, rules for searching / collision searching are pre-protected: for example, these will be two line spaces with the length equal to the size of the cipher block used by the algorithm. One row space is β€œXXXXXXX1”, β€œXXXXXXX2”, β€œXXXXXXX3” and so on. The second is β€œYYYYYYY1”, β€œYYYYYYY2”, β€œYYYYYYY3” and so on. Each element of this space is encrypted and the last 3 bytes are taken from the result. The entire space of the first sequence is stored in RAM. During the calculation of the second sequence, we look for matching elements with the first sequence and remember these collisions. For example, for some given key, the element with sequence number 123 will have the same value of our hash with the element of the second sequence with sequence number 5678: that is, the last three bytes of the encrypted line β€œXXXXX123” will match the last three bytes of the encrypted line β€œYYYY5678”. The stored hashes of the first sequence are our cache. The required cache size (so that, for example, at least 60% of collisions fall) depends on the size of the hash values.

The task itself is:

 {"key": "KEY", "size": 1048576, "count": 4096} 

where KEY is a random encryption key, size is the size of the sequence to cache, and count is the minimum number of collisions to find. The solution is a list of pairs of consecutive sequence numbers, where the collision was found (in our case, one of the pairs is 123 and 5678).

 def solve(key, size, count): bf = Blowfish(key) cache = {} for i in xrange(size): val = str(i) val = "X" * (8 - len(val)) val = bf.encrypt(val) cache[val[-3:]] = i collisions = [] for i in xrange(size): val = str(i) val = "Y" * (8 - len(val)) val = bf.encrypt(val) found = cache.get(val[-3:]) if found: collisions.append((found, i)) if len(collisions) == count * 2: return collisions 

For our task from the server from the example, the client needs to perform 1048576 + 4096 encryption operations (minimum) or 1048576 + 1048576 (maximum) and three megabytes of RAM at least (in practice, a little more to store the list of hashes of the first sequence in the form of a tree to quickly find the presence of element in it). To check, you need to perform 4096 + 4096 encryption operations and have a little more than 4 kilobytes of RAM (overhead for the work of Blowfish).

It is highly desirable that the required number of elements be such that in the vast majority of cases, force the client to go through the entire second sequence (estimate how many collisions there are at a given cache size and take a number around this). It is also very important to estimate the size of the cache: it can be small enough to fit completely into the cache of the processor of the second or even the first level, which will give a significant acceleration of the calculations.

Davier mayer


As mentioned above, platforms that only have JavaScript do not have ready-made cryptographic tools, such as even hash functions. There are often situations where the use of hashes is constantly required due to convenience.

Known common functions such as MD5, SHA1, SHA2 for such platforms will have to implement (or use someone else's implementation) independently. For political reasons, the choice of licensing restrictions can be drastically reduced. More code means more likelihood of errors.

Since we already have a working block cipher, it can be easily applied to data hashing tasks. Most of the hash functions are based on one-way compression functions . The function has two inputs and one output, all of the same size. Upon exiting the function, it is impossible to recover and assume what was at the inputs. A regular block cipher almost has this property and therefore it can be used as a function of compression. This structure is called the Davier-Meier function.

       ───────────────────┐
       β”‚ ∨
     ───┐ β”Œβ”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”
 ──> β”‚ H β”‚ ──> β”‚ E β”‚ ──> β”‚ XOR ──>
     β”€β”€β”€β”˜ β””β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜
                 ∧
                 β”‚
                 β”‚

                 m

The message is divided into parts of m (the length of each is equal to the length of the cipher block), each part is encrypted. Ciphertext H of the previous block is used as the encryption key. The very first iteration uses predefined constants.

 def hasher(data): data += "\x80" if len(data) < 8: pad_length = 8 - len(data) else: pad_length = 8 - len(data) % 8 data += "\x00" * pad_length prev = "1aec98c401022e7c".decode("hex") for i in xrange(0, len(data), 8): bf = Blowfish.new(data[i:i + 8]) prev = strxor(bf.encrypt(prev), prev) return prev 

, (padding) . :



- . Blowfish : 64 . , ( ), 64 ( 128 ). TLS. , , .

- - , . , , - , , .

- SHA2 , AES? - . . : Blowfish . Blowfish - , S- , DES, .

- . - Arcfour Salsa20 . Salsa20 BLAKE -, Salsa20 .


, -, , .

PKI TLS. TLS. ( , curl, ), , TLS , .

, , - JavaScript, , , Ed25519 , padding RSA β€” . , .

: β€” . : ivi , . , , (, CHAP ) , .

MAC (message authentication code) . MAC HMAC ( , Poly1305 ). - ( ). , .

HMAC , . MAC , -. , MD5 , - , HMAC-MD5 . MAC, CBC-MAC , CBC- MAC.

«» CBC-MAC : . , - CBC-, MAC-, , . : . . .

JSON, . . MAC.

 def mac(k, data): data = struct.pack("!I", len(data)) + data if len(data) < 8: pad_length = 8 - len(data) else: pad_length = 8 - len(data) % 8 data += "\x00" * pad_length k = Blowfish.new(k) prev = "\x00" * 8 for i in xrange(0, len(data), 8): prev = bf.encrypt(strxor(data[i:i + 8], prev)) return prev 

MAC PoW , , , Blowfish.

findings



All the best, do not switch!

Β© , Python Go ivi.ru

Source: https://habr.com/ru/post/249359/


All Articles