Primitives
Cages
Evervault Cages are the easiest way to build, deploy and scale Secure Enclaves.
Evervault Cages allow developers to easily deploy Docker containers in a Secure Enclave, powered by AWS Nitro Enclaves. Cages offer easy deployment, invocation and attestation of Secure Enclaves without the engineering overhead.
Cages Trial
How long is the trial program?
Starting from the day you deploy your first Cage, you will have full access to the Evervault Platform (free plan) and Cages for 14 days, so you can try building and deploying Cages. After that period, you’ll have the option to sign up for monthly pricing to continue testing.
If you’re unsure how long you have — you can always check the countdown timer in your Evervault Dashboard to see how much time is left in the allotment. If you need to adjust the timing or have any questions regarding the trial, feel free to reach out to our team.
How many Cages can I create?
During the 14 day trial period — A team can run one Cage. In order to manage our costs, Cages will run as a single instance which may cause downtime for deployments, however this is only for your trial period.
Following initial trial — If you upgrade to a paid plan and start using Cages in production, we will provide multiple instances with high availability and zero downtime deployments.
What happens during the trial?
Try it out! Start here if you’re not sure where to get started.
We would love to know more about the use cases you are building for, any errors or bugs you encounter, and any feedback you have.
- Reach out to support@evervault.com
- Join our Discord server
What happens after the trial?
At the end of the 14-day period, we’ll notify you that your trial access has ended, and any active Cages will be deactivated.
Once deactivated, you will be able to access your Cage in your free account, but no instances will be running. If at any time you’d like to continue using Cages, you can easily upgrade to a paid plan and we will restart your Cage instances.
Upgrading to the paid plan will give you access to all of the Evervault Pro features and allow you to run multiple Cage instances with high availability and zero downtime deployments.
Other questions?
If anything comes up during the trial, please send us an email at support@evervault.com, and we’ll get back to you as soon as possible.
Getting Started
Get up and running with Cages.
Install the Cages CLI
You can install the Cages CLI by running the following command in your terminal:
The Cages CLI is currently separate to the main Evervault CLI.
Install Docker
The Cages CLI needs to perform Docker builds using the Docker CLI. Docker must be installed on your system.
Clone Hello Cage repository
To create your first Cage, you’ll need a server and a Dockerfile
.
If you don’t have one already, then you can use our Hello, Cage!
repository (a simple Node.js Express server) as a starting point. You can clone it by running the following command:
Authenticating with the Cages CLI
The Cages CLI currently only supports API Key authentication. To setup a session in your terminal, you will first need to create an App API Key in the Evervault Dashboard.
Once you have your API Key you can create the session by running:
Initialize your Cage
Now we can initialize your Docker server as a Cage. This will reserve the Cage name within your Evervault App, so you can deploy your service.
For our first Cage, we’ll enable some features which help us see what’s happening inside the Secure Enclave, namely: debug mode and network egress. To do that, we need to run the following command:
Configure your Cage
This command will generate a cage.toml
file in your current directory containing the configuration for your Cage. It should look something like this:
You can edit this .toml
file whenever you want to reconfigure your Cage.
Build your Cage
Now that we have our cage.toml
, we can build our Cage. The build command will convert the service’s Dockerfile
into a Cage-compatible Enclave Image File (.eif
). A .eif
file is a binary image that is used to initialize an AWS Nitro Enclave, and all attestation measures are based on the .eif
binary.
Because our cage.toml
is in the current directory, we can run:
The first build will be slow (approximately 2 minutes) as the CLI has to build several Docker images. Each subsequent build should be faster once the images have been cached.
Once the Cage has finished building, the Cage PCRs will be written to the cage.toml file. This will help with tracking changes in attestation measures within source control.
Our cage.toml
should have a section that looks something like this at the end of it:
These four PCR values are the attestation measurements that we will use to trust the remote server (the Cage) before we share any sensitive data with it. The four measurements reflect various aspects of the image you are running in the enclave.
PCR | Measures | Explanation |
---|---|---|
PCR0 | Enclave Image File | A measure of the Image that will be run within the Cage. |
PCR1 | Linux Kernel and Bootstrap | A measure of the kernel and boot ramfs data. |
PCR2 | User Application | A measure of the User Application without the boot ramfs |
PCR8 | Signing Certificate | A measure of the certificate used to sign the Enclave Image File. |
You can read more about attestation for Cages here.
The build command will also create two files: enclave.eif
and ev-user.Dockerfile
. The enclave.eif
file is the image that the Cage will run in the AWS Nitro Enclave, and the ev-user.Dockerfile
is the Dockerfile
that was used to generate it.
Deploy your Cage
To deploy your Cage using an existing .eif
run:
You can build a fresh .eif
for a deployment by omitting the --eif-path
argument. This means you can skip the ev-cage build
step and deploy in one command.
The CLI will track your Cage deployment. Once the deployment has stabilized, you should see a log which includes the domain for your Cage:
Invoke your Cage
You can now call your Cage over the internet! If you deployed the Evervault Hello, Cage!
template, you can use the following cURL command to try it out:
You should now see an echo response from your Cage:
Reference
cage.toml
Cages allows configuration to be embedded within the source code, so it is easily attestable and can't be tampered with. Simply include a cage.toml
file in the root of your repository and Evervault will automatically include it at build time.
The cage.toml
can be used to configure rules such as network egress and debug mode. It also includes dynamic attestation measures so each time your Cage is built, the PCR measures can be verified by comparing the result of the attestation with the measures included in Evervault's TLS attestation.
The format of your cage.toml
is as follows:
TLS Termination
TLS termination is enabled by default on your Cage. When enabled, any requests into your Cage will terminate TLS within the enclave using a cert signed by the Evervault Root CA. With TLS termination enabled, you will have access to features like automatic field decryption, authentication using your Evervault API Key, and transaction logging. You can disable TLS termination by setting disable_tls_termination=true
in your cage.toml
configuration file. If you disable TLS termination, you will need to handle it within your process.
Cages provide a trusted cert that is provisioned upon startup. This cert is signed by a Trusted CA, making it compatible with most browsers. However, this cert does not include the attestation document, preventing attestation on the trusted domain at this time. Our SDKs are currently being updated with a new attestation flow to handle attestation on trusted certs.
To connect to your cage with the trusted cert use the domain *.cage.evervault.com
(note cage, not cages).
Automatic Decryption
When TLS termination is enabled on your Cage, any Evervault-encrypted strings will be decrypted automatically before being sent to your process. The fields can then be treated as plaintext within your Cage.
Encryption API
Evervault exposes a simple encryption API within Cages which allows you to encrypt and decrypt Evervault-encrypted strings. It also allows you to manually retrieve an AWS Nitro Enclaves Attestation Document if you wish to implement your own attestation protocol.
Encrypt data
To encrypt data, you can make a request to /encrypt
.
Decrypt
To decrypt data you can make another request to /decrypt
.
Attestation Document
To fetch the attestation document for the cage, you can make a request to /attestation-doc
Default port
By default, traffic within the Cage will be forwarded to port 8008 on loopback. This can be configured by supplying an EXPOSE
directive in your Dockerfile.
Note that several ports are reserved within the Cage.
Reserved Ports
Certain ports are reserved within Cages for internal services. In certain cases, these ports are only reserved when a feature is enabled. The reserved ports are as follows:
Port | Reason | Corresponding Feature |
---|---|---|
9999 | Used by the Encryption API | — |
53 | Used to proxy egress DNS calls | egress — can be disabled in the cage.toml |
443 | Used to proxy egress TCP traffic | egress — can be disabled in the cage.toml |
Environment Variables
Environment Variables can be configured using the UI or CLI. Secrets can be encrypted on creation and will be made available to your Cage environment in plaintext on startup. When you create or modify an Environment Variable for your cage, it will only take effect once you redeploy the cage. You can restart your Cage's current deployment to redeploy your Cage without affecting it's PCRs.
Add a secret value
Add a non secret value
HTTP Transaction Logging
When TLS Termination is enabled on your cage, you can enable HTTP Transaction logging which will propagate request logs to the Evervault Activity Screen within the dashboard. These logs contain HTTP Status Codes, Methods, Timestamps etc. This can be configured within the cage toml by setting trx_logging = false
(Default is true
).
A unique ID is also appended to each request. This ID can be found in the response headers under x-evervault-cage-ctx
. You can then search for this ID in the Activity Screen.
By default, your Cage's transaction logs will obfuscate any non-standard HTTP headers. You can specify a list of headers to pass through using the trusted_headers
key in the cage.toml. This can be used to surface error codes or similar diagnostic information from your Cage. For example:
Note: certain headers cannot be trusted to avoid accidentally leaking credentials. This currently includes: api-key
, authorization
, and proxy-authorization
.
Debug Mode
If you want to test your Docker Image in a Cage and see the output, you can run your Cage in debug
mode. Transaction Logs will be emitted and viewable through the Evervault Dashboard. Raw logs from your cage can be viewed using the ev-cage logs
command in the CLI.
When running a Cage in debug
mode, the attestation measure returned will consist entirely of zeroes. This means that a Cage in debug mode is not attestable.
Egress
By default there is no networking out of a Cage. If you wish to make requests out of the Cage, turn on egress by including --egress
on Cage init or update the cage.toml. The default egress port is on port 443 to support typical HTTPS traffic, if you are making requests to hosts on different ports you will need to specify them in the cage.toml or with --egress-ports 443,465,5432
You can also configure the allowed domains that requests out of the Cage should be allowed to with --egress-destinations
. The list will be enforced in both the data plane and the control plane, requests to domains that are not explicity allowed will fail at the beginning of the TLS handshake. You can specify exact domains or wildcards to allow all subdomains, for example cages.evervault.com
or *.evervault.com
. The default behaviour is to allow requests to any domain but we recommend restricting them in production environments.
Egress will only work out of the enclave if the connection uses TLS.
Authentication
Cages expect an Evervault API key in an api-key
request header by default. This can be disabled if you wish to implement your own form of authentication to the Cage.
Healthchecks
By default, Cages will perform a shallow healthcheck. The communication between the host instance and the Enclave will be tested, and the in Enclave environment will be validated to ensure that the Cage initialization completed successfully.
This healthcheck can be extended to include a GET
to the service running in the Cage. Every second, the Evervault-Healthcheck-Agent
will send a GET
request to your process. Your process will be regarded as healthy if it responds with a successful status code (i.e. 2XX).
Cages CLI Reference
Manage Cages from your terminal.
Build
The build command mirrors the docker build
command but produces an Enclave Image File (EIF) as output instead of a Docker image. The build command requires a cage.toml
to be available when creating an Enclave Image File. This can be generated using the ev-cage init
command.
New Certificate
Create a new Cage signing certificate and private key.
Path to the directory where the credentials will be saved. Defaults to the current directory.
Upload Certificate Metadata
Upload a signing certificate's metadata to the Evervault API. This cert can then be used specified for locking deployments.
Path to directory where the signing cert is. Defaults to the path specified in ./cage.toml
Lock Certificates
Interactive prompt to lock Cage deployment to specific signing certificates. A Cage deployment will fail if the signing certificate used is not in specified locked certs. If no certificates are locked to a Cage, it can be deployed with any certificate.
Delete
Delete the Cage defined in a given cage.toml
.
Deploy
Deploy the Cage defined in your cage.toml
file. By default, the deploy command will ignore any prebuilt EIFs and begin a fresh Cage build. You can prevent this by providing a path to an existing EIF using the --eif-path
option.
Path to Dockerfile
for Cage. Will override any Dockerfile
specified in the cage.toml
file
The endpoint to use for healthchecks. See healthchecks for more information.
Restart
Restart the Cage defined in your cage.toml
file. This will redeploy the Cage with the same image as your current Deployment, but with the latest environment variables and secrets. This won't affect your Cage's PCRs
Describe
Get the PCRs of an existing EIF.
Init
Create a Cage and initialize a cage.toml
in the current directory.
Debug setting for the Cage. When debug is enabled, you can access logs from within the Enclave.
Flag to enable cert generation during initialization. This will use the default certificate.
List Cages
List your Cages
List Deployments
List the deployments for your Cage.
Logs
View the logs for your Cage. Only available for Cages in debug mode.
Add Environment Variable
Add environment variable to be used in a Cage.
Delete Environment Variable
Permanently delete a environment variable from the Cage environment.
Get Environment Variables
Get the environment variables in json format.
Encrypt String
Encrypt a string that can be decrypted in your Cage.
FAQ
How does attestation work with Cages?
Evervault Cages abstract away the complexity of implementing and verifying attestation with your Secure Enclaves. Cages embed the attestation flow in the TLS handshake, which is performed every time you invoke a Cage.
TLS Attestation
When you connect to a Cage, your client performs a standard TLS handshake with the enclave. Evervault automatically handles all load balancing between enclaves at the network layer.
Each Cage provisions its own self-signed TLS certificate, with the AWS Nitro Attestation Document embedded within the certificate.
When a client connects to the enclave, it can verify the attestation before completing the TLS handshake. By default, the Evervault server-side SDKs include functionality to verify your Cage's attestation out-of-the-box.
How large can a Cage be?
The current limit for Cages is 16 vCPUs
and 64 gigabytes
of RAM.
Enclaves use a RAM-based filesystem. This means that the memory allocated to your Cage is used for both storage and RAM.
Cages and Evervault Encryption
Evervault Cages allow you to process data encrypted using Evervault Encryption. By default, TLS termination is handled automatically within your Cage, and encrypted data is automatically decrypted before it is passed to your application.
Can I run a Cage on my own infrastructure?
Evervault Cages are managed and run by Evervault on our infrastructure. A major advantage of using Cages is that the burden of hosting and scaling all of the infrastructure necessary to run Secure Enclaves is handled by Evervault.
Evervault running your Cage doesn't weaken the security guarantees provided by Secure Enclaves, thanks to attestation. You are still provided with the attestation measure at build-time and can verify that these haven't been tampered with — all within the TLS handshake.
Can I run multiple instances of a Cage?
Yes! Cages are deployed on two separate EC2 instances split between two Availability Zones in a region for greater resiliency. The number of instances running, and the regions they run in will soon be made configurable.
Can I restart my Cage?
Cages can be restarted from either the "Versions" tab in the Cages Dashboard or by using the CLI.
Why does my Cage perform decrypt requests on start-up?
Cages order a trusted TLS certificate on their first deployment. The private key is encrypted in the Cage, and backed-up so it can be loaded in on every subsequent start-up. As a result, when each instance of the Cage launches, they perform a single decrypt request to load in their private key in plaintext.
Best Practices Building with Cages
Cages are designed to be flexible to fit your development process. Here are a few best practices to set you up for success.
Writing the Dockerfile
Since Cages are built using Docker, a good place to start is making sure that you are formatting your Dockerfile correctly.
First you need to use the FROM
command which will create a layer from the Docker image you wish to use. It’s best to use one of the container images available on Docker Hub to ensure you are using a trusted source. You can also opt for using the slim
version of the container which can help reduce the image size, however the slim
versions may not contain all of the common packages in the default tag.
Next you will need to copy any files from your local source to the filesystem of the container. It’s best to use absolute paths in the Dockerfile. When your Docker container is converted to an Enclave, your commands might not be run in the directory you expect them to be, so it is safer to reference your files as absolute paths.
Make sure that you are exposing the non-reserved port you want to listen on since that is where traffic within the Cage will be forwarded. The Cages documentation uses PORT 8008
as an example, but this can be any available port that you define.
Here are some examples of what your Dockerfile might look like.
Node.js
Java
For Java and other compiled languages, you may need to use a multi-stage build that involves first creating a build stage and then a package stage.
Ruby
Docker will then build images by reading the instructions from your Dockerfile. The default instance size supports up to 2GB Docker images.
You can check the size of your Docker image using the docker inspect
command
Run Your Server Locally with Docker
One helpful step you can take is to try containerizing your application before deploying the container inside a Cage. If it can run with Docker, it should be able to run in a Cage, and it will help you debug errors that may exist from the Docker build before you attempt to run it in a Cage.
Once you have your app ready including the Dockerfile, navigate to the directory containing the Dockerfile, then run the following command:
Next, you can start the container using docker run
along with the name of your image and the container port that you exposed in the Dockerfile mapped to the host port.
Now if you go to the host port http://localhost:8080
you should see your app running. You can also check the Docker Desktop app or CLI to confirm that the container is running.
Package Installation
Some of the instructions in your Dockerfile will install required packages. You can use whichever package manager makes sense for your application.
In some cases, you may want to use a manifest file that will contain the required packages and their versions, like a package.json
for Node or requirements.txt
for Python.
For Python specifically, it can also be helpful to use a virtual environment. You can run the virtual environment within the container and then install the packages within the environment. This will also help with minimizing image size.
Here is an example:
Using the Encrypt and Decrypt Endpoints
By design, data that is encrypted with Evervault that passes into the Cage will automatically decrypt. There is no further action required to decrypt and start using the data.
You also can utilize /encrypt
and /decrypt
endpoints that are exposed on a mini app listening on port 9999
within the enclave. These are not available outside of the enclave, so if you have publicly available endpoints within your app that use the same naming, there should not be any conflict.
These endpoints are available for any Cage, but you will need to define them within your application. Here is an example of how you might implement it in Node.
To encrypt data, you can make a request to /encrypt
and pass in any data that you wish to encrypt as JSON.
The purpose of the decryption API is to offer flexibility with applications built inside the enclave. Examples could be if you need to upload a list of large files of encrypted data to be decrypted on start up of the enclave or a process that pulls in encrypted data from an external source.
Another point to note is that when TLS termination is turned off in the data plane, it’s not possible for the request to be scanned for encrypted strings. This is another instance where the Decrypt API could be used.
Here is an example of how to implement the Decrypt API.
To decrypt data you can make another request to /decrypt
and pass in the encrypted string as JSON.
Should you need to make outbound network calls from within an enclave you can make egress network calls. You will want to disable this at build time in order to prevent unauthorized data from flowing in or out of the Cage. While these options can be helpful for development, you will want to think about which ones should be disabled in production to reduce the attack surface.
Checking Attestation
The Cage Attestation measures are embedded in the TLS extensions within the TLS handshake. Evervault SDKs can be used to validate the PCRs returned in the handshake. You may use the Python or Node.js SDKs which have a function that will allow your application to attest all TLS connections to Evervault Cages.
Node.js
This takes in a set of expected PCRs and validates they are returned in the TLS handshake of requests sent to the Cage. You can pass in a single PCR value as an object or multiple PCR values as an array of objects.
Python
This is the same behavior in Python. You can pass in a single PCR value as a dictionary or multiple PCR values as a list of dictionaries.
Another way is to use openssl using this command:
You can then validate it locally using the ev-cage attest
command in the CLI. This will validate that the PCRs in the cage.toml
file match the PCRs returned in the handshake.
To get the attestation doc itself, you can expose the available endpoint. When running your Cage in debug mode, every value in the attestation document returned will be 0, so if you need to check the attestation measures do not use the --debug
flag when initializing the Cage.
Then make a request to the Cage using the below command.
Utilizing Logging
To turn logging off or on, you will need to define trx_logging = true
or trx_logging = false
in your cage.toml
. You can also leave it undefined and it will be turned on because the default state is set to true
.
From there you should be able to see all of your calls to the Cage in the Cages Dashboard under Activity.
You can also find a unique ID from the request and can use that ID to search for a specific call in case you have many calls to look through.
IP Addresses
There are two ways to see the original IP on the connection.
When using TLS Termination, the IP will be included in the HTTP X-Forwarded-For
header.
If you have TLS Termination disabled, you can receive the client IP via proxy protocol by setting the FORWARD_PROXY_PROTOCOL
environment variable in your Cage.