In this post you will build an app that confidentially takes a passport image and invokes Optical Character Recognition (OCR) on it inside a Enclave. You can follow along by cloning this repo.
There are many cases in which a customer might be required to submit proof of identity — for example, banks and lenders often require customers to provide a passport or driver's licence in order to apply for a loan or a mortgage. This is often to prevent people from opening accounts using false identities.
One way to speed up verification is through the use of Optical Character Recognition (OCR), a process that converts an image of text into a machine-readable text format. There are many available stable OCR libraries and projects that can help with this process in a variety of programming languages.
However, because these documents contain Personally Identifiable Information (PII), they pose a risk to the customer and the institution if a threat actor were to capture them in the processing flow. The check is to make sure the customers are not fraudulent — so the last thing you'd want to do is make them vulnerable to those trying to do just that.
In this guide you will see how you can invoke the OCR process inside of an Evervault Enclave: a Docker container with your app that runs inside a cloud based Trusted Execution Environment (TEE), ensuring that nobody can access the document or any of the extracted PII data. This app will take an uploaded passport document and send it into a Enclave, where you will use a Python OCR library to extract the data and let the customer know whether their application is approved, denied, or requires more information.
- Python version 3.6 or higher (install here)
- An Evervault account (create a free account here)
- Docker installed (get it here)
Clone the working GitHub repo.
To run Enclaves you will need to install the Enclaves CLI by running the following command.
The back end of the app that will run protected inside of the Enclave uses the PassportEye library to identify machine readable zones (MRZ) within an identity document and the pycountry to obtain the name of the country and nationality.
Once you have extracted the data from the document, you will format it into an object with all of the data.
PassportEye offers a measure called validity score where:
- If the score is less than 60 it is likely the document is fradulent
- If the score is moreo than 60 but less than 80, the document could be valid but is suspicious
- If the score is more than 80, the document is considered valid
In this example, you will define that a check has been approved if the validity score is more than 80, requires further review if between 60 and 80, and is denied if less than 60
For security reasons, you will only send back whether the document was approved, denied, or requires further review — you can alter the code if you would like to send back additional data.
Open up the Dockerfile. You will use a virtual environment to install the required libraries needed to run the app. You’ll also tell Docker your webserver will listen on port 8008, which matches the Flask server port defined in app.py.
First, make sure that you have Docker running. Then, get your Evervault API Key from your account.
In your terminal run the following command to initalize the Enclave. You can use the suggested name below or change it to one of your choosing.
You should see that a
enclave.toml are generated. Open up the enclave.toml. You can see that important details are generated that will help with the deployment of your Enclave.
Now build the Enclave using the following command. This will also generate a Dockerfile used to generate the EIF file which will run inside the Enclave (it may take a few minutes).
It will also provide a list of PCRs, cryptographic measurements that are required for the attestation process for the Enclave. These PCRs will be injected into your
You will need the PCRs at a later step when setting up the client in order to make an attested request to the Enclave (you can still make a request to the Enclave without these, it just won’t be attested).
Finally, deploy the Enclave.
Copy the PCR values generated and add them into the empty JSON values that they correspond with in
The code will take the passport image that you upload and pass it into the Enclave. It will also calculate the time elapsed to run the Enclave so that you can monitor it to evaluate for performance.
In order to upload the image as an encrypted file in the
client/templates/form.html file, you will need to provide your app ID and team ID. Add these to the
.env.example file, and rename the file to
Now you can start the Flask server that will run the
client/app.py code and serve the front-end so you can upload a passport image and test the app.
Run the following commands.
Now you can visit
http://127.0.0.1:5000/form and try uploading a passport image to test the app. When working, you should recieve a browser alert with the status of the application.
In this guide, you used Python to invoke OCR on a sensitive identity document, and kept it safe throughout the process. If you ran into an issue or have any questions about this guide, feel free to raise them on GitHub. What other use cases will you use confidential OCR for?