Sign and Verify JWT With Hashicorp Vault REST API

Cryptography is complicated in more than just one way. Therefore, it is commonly recommended not to roll your own, but instead, employ tried and tested methods. Unless you are an experienced cryptographer, it is likely to overlook crucial things, for example, when to authenticate an encrypted message – before decrypting or after? This blog post is about JSON Web Tokens that are digitally signed with an RSA key. Instead of implementing the signing and verification code yourself, you should be using a dedicated server component to do the complex crypto for you, like Hashicorp Vault.

What and Why?

Let me take a few steps back and explain where this is coming from. This is one of those blog posts where I will start with a lot of text and background information before I get into the details. If you only care about signing and verifying JWTs, please skip forward to API to Sign and Verify.

If you are an Azure customer, then you have probably heard of Azure Key Vault, Microsoft’s implementation of a key and secret management service that also performs cryptographic operations. Depending on how many coins you insert into the Azure machine, it is even backed by an HSM. Hashicorp Vault is like Azure Key Vault but open source.

But what is the purpose of such a service? There are several reasons. The main advantage of services like these two is that they relieve your application of managing and storing encryption ciphers or key-pairs. If your application contains a vulnerability that exposes too much sensitive information, the encryption keys are not among them – because it does not know them. Access to these secrets and the operations that can be performed with them can be restricted as well – at least on Azure. I believe Hashicorp Vault has some form of RBAC, too, but I am not as confident as for Microsoft’s Key Vault. There is another benefit as well. Relying on a separate service for all the cryptography is less likely to result in errors in your implementation.

Why choose Hashicorp Vault over Azure Key Vault? Hashicorp Vault is Open Source Software released under the Mozilla Public License, and therefore you are not required to pay for it. It isn’t even limited to the Azure services like Azure Key Vault (after all, it has Azure in the name 🤭). There is another crucial advantage to using Hashicorp Vault: no arbitrary performance constraints. You see, with Azure Key Vault, you are limited to a specific number of cryptographic operations within a certain period. For signing and verification with a 4096 bit RSA key, this amounts to 125 operations per 10 seconds. That could be a problem depending on your workload. Hashicorp Vault does not have this restriction. It can go as fast as your CPU allows, and you can even scale horizontally if clock speed is a limiting factor, but not the number of available cores. The downside, of course, is that you must manage this service yourself instead of simply paying Microsoft to do it for you.

Quick side-note: I am aware that 4096 bits are not more secure than 2048 bits judging from the number’s size. It is for science. 🎓

This performance angle led me to investigate how to sign and verify JSON Web Tokens using Hashicorp Vault and ditch Azure Key Vault for this task. In this blog post, I will concentrate on setting up Hashicorp Vault (short "Vault "from here on going forward) and how to use its API to create an RSA key and use that key to sign and verify a JWT. I will not explain how to manage Vault, install it in a cluster, and scale it. For that, please refer to the official documentation. I will focus on the basic setup steps to get the required functionality and run it on your own dev box for development. I may focus on the performance numbers in a future post, and I might also explain how the API could be seamlessly integrated into the Nimbus Java library. Take this with a grain of salt, though, because I am a Chief Executive Procrastinator at the moment. 😩

Setup Vault

Vault needs a configuration file. For my local system, I use the following.

disable_mlock = true
max_lease_ttl = "876000h"

api_addr = "http://127.0.0.1:8200"

listener "tcp" {
    tls_disable = true
    address = "[::]:8200"
}

storage "file" {
    path = "D:\\Data\\Code\\Vault\\data"
}

You can use a double backslash on Windows or a single forward-slash (Unix style), whatever you prefer. A single backslash will result in an error when you start Vault.

Speaking of starting.

vault server -config config.hcl

For a developer machine, there is an easier way. Vault supports a developer mode that circumvents many security hassles if you quickly want to try something. I am not going this easy route to show you what is essential to know.

Which is another excellent segue to the next step. Before you can do anything with Vault, you must initialize it. This will set up internal encryption and spew out a bunch of unseal keys and a root token.

vault oprator init

Unseal Key 1: BSs...86m
Unseal Key 2: d7D...amr
Unseal Key 3: sdQ...1Ko
Unseal Key 4: n7q...3we
Unseal Key 5: WLD...Ttp

Initial Root Token: s.eL...nWx

I think it is needless to say, but I’ll do it anyway: keep those keys secure! Please read the documentation about sealing and unsealing Vault. There is no need for me to rehash this here. In production, you may want to utilize the auto-unseal feature, for example, in combination with Azure Key Vault. Just because you are not using Azure Key Vault to manage your application’s secrets does not mean you cannot use it for other things.

To make life easier, you may want to set two environment variables first. The vault command you will be using next picks them up.

$env:VAULT_ADDR="http://127.0.0.1:8200"
OR
set VAULT_ADDR="http://127.0.0.1:8200"
OR
export VAULT_ADDR="http://127.0.0.1:8200"

$env:VAULT_TOKEN="s.eL...nWx"
OR
set VAULT_TOKEN="s.eL...nWx"
OR
export VAULT_TOKEN="s.eL...nWx"

The last step before you can finally do something with Vault is to unseal it. To do that, execute the vault operator unseal command at least three times and pass it a different "Unseal Key" when it prompts for one. Once Vault is unsealed, the output will look like this.

Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.7.0-rc1
Storage Type    file
Cluster Name    vault-cluster-67266738
Cluster ID      5978e266-4f19-72fc-d6a2-4736ee81d5eb
HA Enabled      false

Interlude

The CLI examples that follow assume that you are running in a Linux environment. I am saying the because Windows users must escape the quotes (") of the JSON body value differently.

# Linux Bash
-d '{"type":"rsa-4096"}'

# Windows Terminal
-d "{\"type\":\"rsa-4096\"}"

If you do not do that, you will get funny results. You can use WSL2 to run a Linux shell on your Windows computer, of course. Then you are all set.

Create an RSA Key

Vault is set up and ready to go. Now you need an RSA key. Before you can do that, you must enable a so-called secrets engine (additional link). To sign and verify data with an RSA key, you can use the transit engine.

vault secrets enable transit

It is time to create the key. The transit engine supports a lot of them. For this exercise, you choose one of the RSA variants, rsa-4096 in particular – for 🎓.

curl -i 
    -H "X-Vault-Token: " 
    -d '{"type":"rsa-4096"}' 
    -X POST "http://127.0.0.1:8200/v1/transit/keys/sign_key"

It is crucial to know that the key-type must be passed in the request body. The API is not 100% clear about it, at least not in prose, and if you attempt to give type as a URL parameter, the request succeeds, but Vault will not create the key you want.

Because this has happened to me, I have also learned what you must do to delete a key (call me Segue King). Vault will not let you delete the key willy-nilly. First, you must tell Vault that the key is deletable, and after that, you can remove it.

curl -i \
    -H "X-Vault-Token: " 
    -d '{"deletion_allowed":true}' 
    -X POST "http://127.0.0.1:8200/v1/transit/keys/sign_key/config"
curl -i 
    -H "X-Vault-Token: " 
    -X DELETE "http://127.0.0.1:8200/v1/transit/keys/sign_key"

The error you will receive when you try to sign data with this unwanted key looks like this.

{"errors":["key type aes256-gcm96 does not support signing"]}

API to Sign and Verify

Finally, this is where it gets meaty. After this wall of text and tedious setup commands, let me present the sign and verify API.

Actually, how about some sample data first? 😜

A JWT is comprised of a JSON header and a JSON body that contains the claims. Base64-encode both objects individually and concatenate the results with a "." and you receive the piece of data that will be signed. Follow this link for a more comprehensive explanation.

My sample data looks like this.

# Header
{
    "kid": "keyName_keyVersion",
    "alg": "RS256"
}
# Claims
{
    "sub": "the-codeslinger",
    "status": "procrastinating",
    "iat": 1593580911
}

As Base64 string:

eyJraWQiOiJrZXlOYW1lX2tleVZlcnNpb24iLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0aGUtY29kZXNsaW5nZXIiLCJzdGF0dXMiOiJwcm9jcmFzdGluYXRpbmciLCJpYXQiOjE1OTM1ODA5MTF9

Now begins the fun part – which is also the shortest part. Sign the token! "Base64" in the curl command does not refer to the "Base64 string" as shown above. Vault can sign anything, also binary data. To transport binary data over a text API like this, the data must first be Base64 encoded. It just so happens that the JWT is already Base64 encoded – albeit with a "." in the middle. Therefore, Base64-encode it again and you will get this.

ZXlKcmFXUWlPaUpyWlhsT1lXMWxYMnRsZVZabGNuTnBiMjRpTENKaGJHY2lPaUpJVXpJMU5pSjkuZXlKemRXSWlPaUowYUdVdFkyOWtaWE5zYVc1blpYSWlMQ0p6ZEdGMGRYTWlPaUp3Y205amNtRnpkR2x1WVhScGJtY2lMQ0pwWVhRaU9qRTFPVE0xT0RBNU1URjk=

Sign it.

curl -i 
    -H "X-Vault-Token: " 
    -d '{"hash_algorithm":"sha2-256","signature_algorithm":"pkcs1v15","input":""}' 
    -X POST "http://127.0.0.1:8200/v1/transit/sign/sign_key"

The signature is returned in a JSON object under the key data.signature. Note that you must remove the vault:v1: prefix before you append this signature to the token.

{
    "request_id": "fd2cd5da-f058-aaf9-4b80-c2bb05fce101",
    "lease_id": "",
    "renewable": false,
    "lease_duration": 0,
    "data": {
        "key_version": 1,
        "signature": "vault:v1:On4...E0="
    },
    "wrap_info": null,
    "warnings": null,
    "auth": null
}

Lastly, verify the signature. This request requires the Base64 encoded JWT as input and the exact value that signing the token returned; that is, the prefix vault:v1 must be prepended to the signature.

curl -i
    -H "X-Vault-Token: " 
    -d '{"hash_algorithm":"sha2-256","signature_algorithm":"pkcs1v15","input":"Base64","signature":"vault:v1:RQF...4I="}' 
    -X POST "http://127.0.0.1:8200/v1/transit/verify/sign_key"

The result can be found at data.valid.

{
    "request_id": "5b1eeb96-03d2-bcf4-b5bd-6fd615d1bb3c",
    "lease_id": "",
    "renewable": false,
    "lease_duration": 0,
    "data": {
        "valid": true
    },
    "wrap_info": null,
    "warnings": null,
    "auth": null
}

Congratulations, you have survived an onslaught of text and CLI commands.

Famous Last Words

The resulting final token looks like the following. To get that, you Base64 decode the result of the signing process (minus vault:v1:) and encode it Base64 URL-Safe. Then you can append it to the JWT with a ".".

eyJraWQiOiJrZXlOYW1lX2tleVZlcnNpb24iLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0aGUtY29kZXNsaW5nZXIiLCJzdGF0dXMiOiJwcm9jcmFzdGluYXRpbmciLCJpYXQiOjE1OTM1ODA5MTF9.On4VIRT6XqTBFjLOAFJXEp7BMn9stnNf9p4e93foCZ_KuIneOKbgYyvoXhVV9_No8QB4--py2WX09cYSY9FtuFviS6l9BsUPQzW5A-YuOmA64kPHYmmbP6APhUzenwMivUaoteB9kVIYGiRTy2DGR0J8iKmS5368GX0VXQw5z5-R7of1r90OCdbcIztnsBTzzsbEvtnob8wicxPY5Qps_l4HgHFlZim9A8xyrRmobjaNrk3fEIYpsfSPyarhoWdu4sTAc79LcdiPVyhNIb6DZ4c6x8McUcUpnOGeLszgAXO59s1Iwax2uc7DO_7PrF_FTr7K6xyqd3Pp0b0XGRjEvWJwy6LUFPB98BEdbsVCq_zrFJYrhL2kjh0dVZzmrJ97O9Wo9_GO3OjREtIoxm3-2Aan39Y09aOHvai6QsbHqsyXpAWUgplMcDDr8GVLDVDtm-6fgISQm46skG_N1dXQIMC_AURtKr2vN49fLzhGQV3UDLgKSXVRV-yNR1m2tfFOmZiy5TCl-CINwhslTNLhHtpD-sYTJDFKcGRkjWjdR98jXLI_0qqUUFk7PVmuBcD4aH6UvW7dq8qDOMMqKRujmk6Bn7i1VzlPRqiy2u7ajI-Txb9dfjWlJnLeQ0sUY2fhWeWNGLU0Nf-_oN8bmPmjl99-A3KzHozRXkErYimXVE0

When you receive a JWT from wherever you receive them in your application, remove the signature, convert it from Base64 URL-Safe to Base64, prefix it with vault:v1: and use that as the value of signature in the verification process. The input value is a Base64 version of the first two token parts.

Note: I have not tested if Vault is also capable of handling Base64 URL-Safe data. Maybe this conversion is not even necessary. I will leave that up to the reader.

I am explicitly saying this because these simple examples reference each other’s output, but in a live application, you take data from wherever it comes from. That means the input to the signing process must not be kept around as a reference for verification. You verify what you receive, for example, at your REST endpoints in a web application. I just wanted to make that implicit in case it came across the other way.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.