Invoke OCR on a Sensitive Document using Python and Cages

There are many cases in which a customer might be required to submit proof of identity — for example, banks and lenders often require customers to provide a passport or driver's licence in order to apply for a loan or a mortgage. This is often to prevent people from opening accounts using false identities.

One way to speed up verification is through the use of Optical Character Recognition (OCR), a process that converts an image of text into a machine-readable text format. There are many available stable OCR libraries and projects that can help with this process in a variety of programming languages.

However, because these documents contain Personally Identifiable Information (PII), they pose a risk to the customer and the institution if a threat actor were to capture them in the processing flow. The check is to make sure the customers are not fraudulent — so the last think you'd want to do is make them vulnerable to those trying to do just that.

In this guide you will see how you can invoke the OCR process inside of an Evervault Cage: a Docker container with your app that runs inside a cloud based Trusted Execution Environment (TEE), ensuring that nobody can access the document or any of the extracted PII data. This app will take an uploaded passport document and send it into a Cage, where you will use a Python OCR library to extract the data and let the customer know whether their application is approved, denied, or requires more information.

Prerequisites

• Python version 3.6 or higher (install here)
• An Evervault account (create a free account here)
• Docker installed (get it here)

Set up

Clone the working GitHub repo.

1
git clone https://github.com/evervault/examples
2
cd cages-ocr

To run Cages you will need to install the Cages CLI by running the following command.

1
curl https://cage-build-assets.evervault.com/cli/install -sL | sh

The Python app

The back end of the app that will run protected inside of the Cage uses the PassportEye library to identify machine readable zones (MRZ) within an identity document and the pycountry to obtain the name of the country and nationality.

Once you have extracted the data from the document, you will format it into an object with all of the data.

1
@app.route('/upload', methods=['GET', 'POST'])
2
def upload():
3
    app.logger.info('upload received')
4
    passport_image = Image.open(request.files['upload'].stream)
5
    data = do_mrz(passport_image)
6
    
7
    return data['applicationStatus']
8
9
def get_validity(score):
10
    if score < 60:
11
        return "Fraudulent"
12
    elif score < 80:
13
        return "Suspicious"
14
    else:
15
        return "Valid"
16
17
def get_status(score):
18
    if score < 60:
19
        return "Denied"
20
    elif score < 80:
21
        return "Further Review Required"
22
    else:
23
        return "Approved"
24
25
def do_mrz(filepath):
26
27
    mrz = read_mrz(filepath)
28
    mrz_data = mrz.to_dict()
29
30
    document_type = TYPE_MAP[mrz_data['type'][0]]
31
    names = mrz_data['names']
32
    surname = mrz_data["surname"]
33
    nationality = pycountry.countries.search_fuzzy(mrz_data['nationality'])[0].name
34
    country = pycountry.countries.search_fuzzy(mrz_data['country'])[0].name
35
    dob = format_dob(get_dob(mrz_data["date_of_birth"]))
36
    valid_dob = mrz_data["valid_date_of_birth"]
37
    valid_expiry_date = mrz_data["valid_expiration_date"]
38
    valid_number = mrz_data["valid_number"]
39
    validity_score = mrz_data["valid_score"]
40
41
    return_doc = {
42
        "applicationStatus":get_status(validity_score), 
43
        "DocumentType": document_type,
44
        "Name": names,
45
        "Surname": surname,
46
        "Nationality": nationality,
47
        "IssuerCountry": country,
48
        "DateOfBirth": dob,
49
        "ValidDateOfBirth": valid_dob,
50
        "ValidExpirationDate": valid_expiry_date,
51
        "ValidNumber": valid_number,
52
        "Validity": get_validity(validity_score),
53
        "ValidityScore": validity_score,
54
        "Sanctions": {}
55
    }
56
57
    print(return_doc)
58
    return return_doc
59
60
def get_dob(dob):
61
    n = 2
62
    return [dob[i:i+n] for i in range(0, len(dob), n)]
63
64
def format_dob(dob):
65
    return f"{dob[2]}-{dob[1]}-{dob[0]}"

PassportEye offers a measure called validity score where:

• If the score is less than 60 it is likely the document is fradulent
• If the score is moreo than 60 but less than 80, the document could be valid but is suspicious
• If the score is more than 80, the document is considered valid

In this example, you will define that a check has been approved if the validity score is more than 80, requires further review if between 60 and 80, and is denied if less than 60

For security reasons, you will only send back whether the document was approved, denied, or requires further review — you can alter the code if you would like to send back additional data.

The Dockerfile

Open up the Dockerfile. You will use a virtual environment to install the required libraries needed to run the app. You’ll also tell Docker your webserver will listen on port 8008, which matches the Flask server port defined in app.py.

1
FROM python:3.8-slim
2
3
# create a virtual environment
4
RUN python3 -m venv /opt/venv
5
# install the requirements to the virtual environment
6
COPY requirements.txt requirements.txt
7
RUN /opt/venv/bin/pip install -r requirements.txt 
8
9
COPY app.py /app.py
10
11
# this must match the port your Flask server in app.py is running on
12
EXPOSE 8008
13
# use the python binary in the virtual environment we've set up to run our server
14
ENTRYPOINT ["/opt/venv/bin/python", "/app.py"]

Initialize the Cage

First, make sure that you have Docker running. Then, get your Evervault API Key from your account.

In your terminal run the following command to initalize the Cage. You can use the suggested name below or change it to one of your choosing.

1
export EV_API_KEY='Your API Key Here'
2
3
ev-cage init -f ./Dockerfile \
4
--name ocr-cage \

You should see that a cert.pem, key.pem and cage.toml are generated. Open up the cage.toml. You can see that important details are generated that will help with the deployment of your Cage.

Build the Cage

Now build the Cage using the following command. This will also generate a Dockerfile used to generate the EIF file which will run inside the enclave (it may take a few minutes).

1
ev-cage build --write --output .

It will also provide a list of PCRs, cryptographic measurements that are required for the attestation process for the enclave. These PCRs will be injected into your cage.toml.

Deploy the Cage

Finally, deploy the Cage.

1
ev-cage deploy --eif-path ./enclave.eif

Setting up the Client

Copy the PCR values generated and add them into the empty JSON values that they correspond with in client/app.py.

1
APP_ID = os.environ.get('APP_ID')
2
API_KEY = os.environ.get('EV_API_KEY')
3
CAGE_NAME = os.environ.get('CAGE_NAME')
4
5
evervault.init(APP_ID, API_KEY)
6
7
@app.route('/form', methods=('GET', 'POST'))
8
def form():
9
    if request.method == 'POST':
10
        uploaded_file = request.files['myFile']
11
        filename = uploaded_file.filename
12
        if filename != '':
13
            file_path = os.path.join(app.config['UPLOAD_PATH'], filename)
14
            uploaded_file.save(file_path)
15
16
        attested_session = evervault.cage_requests_session({ 
17
            CAGE_NAME: {
18
                "PCR0": "",
19
                "PCR1": "",
20
                "PCR2": "",
21
                "PCR8": ""
22
        } 
23
        })
24
25
        with open(file_path, "rb") as file:
26
            file_bytes = file.read()
27
28
        fields = {
29
            "upload": (filename, file_bytes),
30
        }
31
32
        body, header = urllib3.encode_multipart_formdata(fields)
33
34
        start = time.time()
35
36
        response = attested_session.post(
37
            f'https://{CAGE_NAME}.{APP_ID}.cages.evervault.com/upload',
38
            headers={'Content-Type': header},
39
            data=body
40
        )
41
42
        print(f'Time: {time.time() - start}')
43
        print(response)
44
        return jsonify(response)
45
    return render_template('form.html', team_id=os.environ.get('TEAM_ID'), app_id=os.environ.get('APP_ID')

The code will take the passport image that you upload and pass it into the Cage. It will also calculate the time elapsed to run the Cage so that you can monitor it to evaluate for performance.

Add Environment Variables

In order to upload the image as an encrypted file in the client/templates/form.html file, you will need to provide your app ID and team ID. Add these to the .env.example file, and rename the file to .env.

1
EV_API_KEY=
2
TEAM_ID=
3
APP_ID=

Run the Client

Now you can start the Flask server that will run the client/app.py code and serve the front-end so you can upload a passport image and test the app.

Run the following commands.

1
python3 -m venv venv
2
source venv/bin/activate
3
pip3 install -r requirements.txt
4
cd client
5
flask run

Now you can visit http://127.0.0.1:5000/form and try uploading a passport image to test the app. When working, you should recieve a browser alert with the status of the application.

Conclusion

In this guide, you used Python to invoke OCR on a sensitive identity document, and kept it safe throughout the process. If you ran into an issue or have any questions about this guide, feel free to raise them on GitHub or drop them in the Evervault Discord. What other use cases will you use confidential OCR for?