Formation of JWS and JWK from rsa keys using the example of Let's Encrypt and ISPmanager integration
Hello! My name is Dmitry Smirnov, I am a developer from ISPsystem and it is me who is responsible for the appearance of integration with Let's Encrypt in the ISPmanager 5 panel. I'll tell you how the plug-in was developed, how it changed and how it came to its present state. From the text, you will learn how to form JWS and JWK from rsa keys and get Let's Encrypt a certificate for ACME v01. If interested, welcome under cat.
The first version of the plugin was beautiful in every way. Her success is comparable only with her monumental collapse (yes, the writers of The Matrix also read this article). Then they set me the task like this: here's letsencrypt.org , do something with it. Well, I did.
Initially, the plugin simply pulled the official client of Let's Encrypt from the github and worked directly with it. And he, to put it mildly, was not user friendly. No domain? We do not order anything. Pseudonyms are not resolved? Round out. All the preparatory work for issuing the certificate fell on the shoulders of the user, and any mistake led to the unsuccessful receipt of the certificate.
')
Needless to say, the plugin has been returned for revision. So began my fascinating journey into the wonderful world of Internet security and customer focus.
Before the second approach to development, we formulated several tasks:
Of course, the main challenge for me was the first item. Armed with official documentation, I began to develop.
Let's Encrypt ( LE ) was created by the Internet Security Research Group ( ISRG ). Especially for him, ISRG developed the Automatic Certificate Management Environment ( ACME ) protocol. By itself, the process of obtaining a certificate is a POST request to the LE service, where the request body is represented as JSON wrapped in JSON Web Signature ( JWS ).
The steps for getting are:
Let's start in order.
To create and authorize a user, you need a pair of rsa keys in pem-format, which will later serve as the basis for constructing JWS .
The data structure of the POST request to communicate with ACME v01:
Here it is worth focusing on three things. First, Replay-Nonce returns in response headers acme-v01.api.letsencrypt.org/directory . Secondly, Payload is JSON, in which you explain what, in fact, you want from ACME in this particular case. Thirdly, JWS is JSON of the following type (I’ll make a reservation that there are ways to get a signature by other algorithms. Here’s just one, perhaps the simplest):
There was a question where to get data for JWK . Short searches on the Internet have borne fruit, and I have found a simple way to look at the very pair of pem-keys in decrypted form. Here is an example:
output of the command in the shortest possible form:
Here they are, so we need the data, take it - I do not want. I took, led to the right mind, created JWS . But I was in for a cruel disappointment: the signature was wrong. All this resulted in several long hours of searching for information on the Internet, debugging and hopelessness. Still, the answer has surfaced.
It turned out that the first two zeros are artifacts that appear when encoding an integer using ASN.1, but there is another way to get the modulus ready for processing and insertion into JWS .
Voila:
Now let's go through the requests and payloads. First of all, let's call GET for acme-v01.api.letsencrypt.org/directory . From received JSON
take the address of the user agreement and addresses for certificate requests.
Now for each domain name for which we issue a certificate, you need to pass authorization. In response, we get a list of available checks for ownership of the name.
Let's start checking domain names. For ACME v01 version there are three ways available: http, dns, tls. Our choice fell on the first method, as the most simple and affordable. The point is simple: a .well-known / acme-challenge subdirectory should be created in the domain directory, where a verification token will be put - a file with the name specified in the verification.
The token itself must contain the string token_name. Base64Url (fingerprint_jwk) - this will be the so-called authorization key. It is easy to get a fingerprint using OpenSSL with the command:
I'm afraid for bash you have to write the base64url function yourself.
No matter how easy it was, I managed to get stuck for a few hours. Children's mistakes - the worst. In openssl at the end of the JWK , a newline character was transmitted. Be attentive to this data, the footprint must be clean ”:).
The request body will look like this:
and the url is taken from JSON verification.
I wrote a clever mechanism that threw the tokens into the right nodes of the cluster and then deleted them, which later turned out to be an extra work (more on this later).
And, lo and behold, the first LE certificate was obtained successfully!
It remained to solve the problems that awaited ordinary users. How to bypass the inevitable simultaneous release of a certificate when domain aliases are not yet resolved? We decided that the user should receive a certificate immediately when ordering, but self-signed.
We issue a self-signed certificate, we join the domain and register an internal order for a certificate from LE. Every 5 minutes we begin the procedure for receipt. If it fails, calmly wait for the next attempt. We give the user 24 hours to resolve all possible problems, and only then give up and eliminate the certificate from the issuance queue.
The ready certificate from LE is fresh and put in place of the old self-signed one. That's all. This is how the plug-in integration with Let's Encrypt saw the light.
We tried to foresee all the problems that could arise when issuing a certificate, but some errors still eluded our inquisitive gaze. The main problem was the numerous and ubiquitous .htaccess configuration files. Very often they led to a situation where the verification token, carefully placed in the domain directory, simply simply was unavailable. And the only way out for the user was to temporarily disable his settings.
A few months later it became clear that the mechanism of distribution of tokens to the domain directories did not justify itself. For all the web domain panels created by the tools, we started adding the alias /.well-known/acme-challenge/, leading to the / usr / local / mgr5 / www / letsencrypt directory. It was to her that the tokens began to be placed to verify that further reduced access errors to a minimum.
Checking through TXT records in the domain zone appeared just six months ago. Virtually no tricks to prepare arose, except for one. For TXT records, the string that we recorded in the token must be run through the command
That's all for today. About the transition of the plug-in to ACME v02 and support for wildcard certificates in the next release.
Let's Encrypt 1.0
The first version of the plugin was beautiful in every way. Her success is comparable only with her monumental collapse (yes, the writers of The Matrix also read this article). Then they set me the task like this: here's letsencrypt.org , do something with it. Well, I did.
Initially, the plugin simply pulled the official client of Let's Encrypt from the github and worked directly with it. And he, to put it mildly, was not user friendly. No domain? We do not order anything. Pseudonyms are not resolved? Round out. All the preparatory work for issuing the certificate fell on the shoulders of the user, and any mistake led to the unsuccessful receipt of the certificate.
')
Needless to say, the plugin has been returned for revision. So began my fascinating journey into the wonderful world of Internet security and customer focus.
Let's Encrypt 2.0
Before the second approach to development, we formulated several tasks:
- Implement certificate acquisition at the ACME protocol level.
- Inform the user in detail about the process.
- Make it possible to obtain a certificate when creating a web domain.
- Expect rezolv domain names within days after the start of the issuance process.
Of course, the main challenge for me was the first item. Armed with official documentation, I began to develop.
Let's Encrypt ( LE ) was created by the Internet Security Research Group ( ISRG ). Especially for him, ISRG developed the Automatic Certificate Management Environment ( ACME ) protocol. By itself, the process of obtaining a certificate is a POST request to the LE service, where the request body is represented as JSON wrapped in JSON Web Signature ( JWS ).
The steps for getting are:
- check in,
- authorization and receipt of methods to confirm domain ownership,
- proof of ownership
- obtaining a certificate.
Let's start in order.
User registration and authorization
To create and authorize a user, you need a pair of rsa keys in pem-format, which will later serve as the basis for constructing JWS .
openssl genrsa -out private.pem 2048 The data structure of the POST request to communicate with ACME v01:
{ "header": jws, //JSON Web Signature "protected": Base64Url(jws + Replay-Nonce), //Nonce — "payload": Base64Url(payload), // "signature": Base64Url(sign(protected.payload, private.pem)) // } Here it is worth focusing on three things. First, Replay-Nonce returns in response headers acme-v01.api.letsencrypt.org/directory . Secondly, Payload is JSON, in which you explain what, in fact, you want from ACME in this particular case. Thirdly, JWS is JSON of the following type (I’ll make a reservation that there are ways to get a signature by other algorithms. Here’s just one, perhaps the simplest):
{ "alg" : "RS256", "jwk" : { //JSON Web Key "kty" : "RSA", // key type -- "e" : "...", // HexToBase64UrlEnodedByte "n" : "..." // modulus HexToBase64UrlEnodedByte } } There was a question where to get data for JWK . Short searches on the Internet have borne fruit, and I have found a simple way to look at the very pair of pem-keys in decrypted form. Here is an example:
openssl rsa -text -noout < private.pem output of the command in the shortest possible form:
Private-Key: (2048 bit) modulus: 00:a8:c5:cc:9c:24:9b:d1:8d:9a:67:81:4d:1f:57: ... 8c:45:51:9e:26:fc:12:35:9e:a0:10:fd:80:94:cc: 09:a5 publicExponent: 65537 (0x10001) privateExponent: ... prime1: ... prime2: ... exponent1: ... exponent2: ... coefficient: ... Here they are, so we need the data, take it - I do not want. I took, led to the right mind, created JWS . But I was in for a cruel disappointment: the signature was wrong. All this resulted in several long hours of searching for information on the Internet, debugging and hopelessness. Still, the answer has surfaced.
It turned out that the first two zeros are artifacts that appear when encoding an integer using ASN.1, but there is another way to get the modulus ready for processing and insertion into JWS .
openssl rsa -noout -modulus < private.pem Voila:
Modulus=A8C5CC9C249BD18D9A67814D1F57...8C45519E26FC12359EA010FD8094CC09A5 Getting a certificate
Now let's go through the requests and payloads. First of all, let's call GET for acme-v01.api.letsencrypt.org/directory . From received JSON
{ "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change", "meta": { "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf" }, "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz", "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert", "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg", "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert", "zH_Sr0qwmwM": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417" } take the address of the user agreement and addresses for certificate requests.
check in
url = directory["new-reg"] payload = { "resource": "new-reg", "agreement": directory["meta"]["terms-of-service"] } Authorization
Now for each domain name for which we issue a certificate, you need to pass authorization. In response, we get a list of available checks for ownership of the name.
url = directory["new-authz"] payload = { "resource": "new-authz", "identifier": { "type":"dns", "value": "name" } } Checks
Let's start checking domain names. For ACME v01 version there are three ways available: http, dns, tls. Our choice fell on the first method, as the most simple and affordable. The point is simple: a .well-known / acme-challenge subdirectory should be created in the domain directory, where a verification token will be put - a file with the name specified in the verification.
The token itself must contain the string token_name. Base64Url (fingerprint_jwk) - this will be the so-called authorization key. It is easy to get a fingerprint using OpenSSL with the command:
echo jwk | openssl dgst -sha256 -binary | base64url I'm afraid for bash you have to write the base64url function yourself.
No matter how easy it was, I managed to get stuck for a few hours. Children's mistakes - the worst. In openssl at the end of the JWK , a newline character was transmitted. Be attentive to this data, the footprint must be clean ”:).
The request body will look like this:
payload = { "resource": "challenges", "keyAuthorization": } and the url is taken from JSON verification.
I wrote a clever mechanism that threw the tokens into the right nodes of the cluster and then deleted them, which later turned out to be an extra work (more on this later).
Certificate issuance
url = directory["new-cert"] payload = { "resource": "new-cert", "csr": csr } And, lo and behold, the first LE certificate was obtained successfully!
Customer focus
It remained to solve the problems that awaited ordinary users. How to bypass the inevitable simultaneous release of a certificate when domain aliases are not yet resolved? We decided that the user should receive a certificate immediately when ordering, but self-signed.
We issue a self-signed certificate, we join the domain and register an internal order for a certificate from LE. Every 5 minutes we begin the procedure for receipt. If it fails, calmly wait for the next attempt. We give the user 24 hours to resolve all possible problems, and only then give up and eliminate the certificate from the issuance queue.
The ready certificate from LE is fresh and put in place of the old self-signed one. That's all. This is how the plug-in integration with Let's Encrypt saw the light.
Difficulties
We tried to foresee all the problems that could arise when issuing a certificate, but some errors still eluded our inquisitive gaze. The main problem was the numerous and ubiquitous .htaccess configuration files. Very often they led to a situation where the verification token, carefully placed in the domain directory, simply simply was unavailable. And the only way out for the user was to temporarily disable his settings.
A few months later it became clear that the mechanism of distribution of tokens to the domain directories did not justify itself. For all the web domain panels created by the tools, we started adding the alias /.well-known/acme-challenge/, leading to the / usr / local / mgr5 / www / letsencrypt directory. It was to her that the tokens began to be placed to verify that further reduced access errors to a minimum.
DNS verification
Checking through TXT records in the domain zone appeared just six months ago. Virtually no tricks to prepare arose, except for one. For TXT records, the string that we recorded in the token must be run through the command
echo _ | openssl dgst -sha256 -binary | base64url That's all for today. About the transition of the plug-in to ACME v02 and support for wildcard certificates in the next release.
Source: https://habr.com/ru/post/354420/
All Articles