Securing APIs requires the generation of API keys. These api keys will be used in the CLI and SDK versions of your service passed to the REST or gRPC api via the Authorization header. The generation of API keys examples can be found in the article for Creating API Keys. If the generation of API keys is the service side action, then the storage of these api keys is the client side responsibility. Here we'll discuss the best solution for storage of these keys.
API secrets go by many names, CLIENT_SECRET, PASSWORD, API_KEY, ACCESS_KEY, and the list goes on. These may be used to access an API, log in to an existing service, or verify symmetric encrypted data blobs. In almost all cases these should be asymmetric encrypted secrets never stored by the credentials server. (If a service allows you to reveal or display a secret more than once, it is a red flag that it isn't well secured in the service.)
API keys generated for a service client hold all the authority in an application, they can represent a user, a group, full access to API service resources, or be used to impersonate an admin user. And as such the secure storage of these keys is critical. One prevalent pattern is storing the credentials on the client in unencrypted form on the production container. This is not safe however. While the location is of the better ones available, the deployment of unencrypted api keys is rife with security issues. How do you get unencrypted credentials on a target container without exposing those credentials to a system that shouldn't have access? And further who has seen these unencrypted credentials? Limiting exposure of credentials is hugely important to security.
A better solution than this is resolving these credentials at runtime, so the unencrypted form is only available in memory in the target service. Even as such there are two ways to resolve this information:
- Reference by secret name
- Reference by encrypted credentials
Using a secret name
Credentials can be stored in a vault or other secure storage and then loaded from that storage using the name of the secret. The usage of roles and other access control via cloud providers can be used to store the credentials and then securely resolve them at runtime.
await vault.createSecret('SecretName', 'SECRET_API_KEY');const secret = await vault.getSecret('SecretName');
This is a great solution for the storage of the secret, but requires generating a secret name, and potentially exposes this concept other places. It also encourages the reuse of secrets by other services, which violate microservices if that is a pattern being used. In some cases this a good pattern, but picking one that doesn't encourage the reuse of the secret is best.
Another possible problem with this pattern, is that it lacks transparency of secret changes. In the case you will want rotate the secret key, the tracking of the rotation happens outside the purview of the service, and therefore is difficult to track.
Using encrypted credentials
Instead of storing the credentials in another service, a more powerful and secure pattern is to encrypt the credential and store it in the source code. This prevents exposure of the encrypted credentials directly, by limiting visibility to the service's source. While anything can copy the credentials, it's clear that the service owns them and won't be directly used without something copying that full encrypted text. To generate and resolve the credentials at runtime:
const encryptedCredentials = encryptor.encrypt('SECRET_API_KEY');const secret = encryptor.decrypt(encryptedCredentials);
Additionally this solves the problem of audit, since the version control system now tracks exactly when the credential has been updated. Access control is still enforced by similar mechanisms when using the secret name, but the scope of the credential is limited in space to the service where the encryptedCredentials are stored. Rotating the key is as simple as updating the encrypted credentials blob, and the change is transparent for everyone. It also allows handling multiple changes without needing the use of complicated mechanisms like secret key versions. The version is the one in the source!
Comprehensive review of credential security options
This was a conference talk given at Codemotion Milan in 2023: