The Sensitive Data integration enables:
- new data scopes for accessing sensitive data in its encrypted or redacted forms.
- new Jinja2 filters to encrypt, decrypt, and redact sensitive data.
- new Python APIs for accessing entries in their encrypted or redacted forms, and methods for encrypting, decrypting, and redacting sensitive data.
This guide assumes you’ve set up the Sensitive Data integration. If you haven’t done that yet, check out the set up guide first:
In this guide, we’ll create a component that generates a random number and flags it as
sensitive. We’ll store the number on the data scopes, then test out different ways of viewing the encrypted, decrypted, and redacted versions of the number.
The Sensitive Data integration introduces new data scopes:
Used for accessing encrypted sensitive data stored on
Used for accessing redacted (★★★★★) sensitive data on
First, let’s create the random number generator component.
In your app’s
component folder, create a subfolder called
sensitive_data. Inside that folder, create a file called
secret_number.py and copy this code into it:
from dataclasses import dataclass from meya.component.element import Component from meya.element.field import element_field from meya.element.field import response_field from meya.entry import Entry from random import randint from typing import List @dataclass class SecretNumberComponent(Component): @dataclass class Response: result: int = response_field(sensitive=True) sensitive: bool = element_field(default=True) from_: int = element_field() to: int = element_field() async def start(self) -> List[Entry]: secret = randint(self.from_, self.to) return self.respond(data=self.Response(secret))
This component will generate a random number and store it at
Notice that on line
14we’ve flagged the
sensitive. This means that if the Sensitive Data integration is enabled with
transcript: trueand the entry generated by this component is flagged as
resultwill be available in its encrypted and redacted forms.
In this case, we are forcing the entry to be flagged as sensitive by setting the component’s
Trueby default (line
Now, we’ll create a flow that uses the random number generator and demonstrates what the secret number looks like on the different scopes.
In your app’s
flow folder, create a subfolder called
sensitive_data. Inside the new folder, create a file called
scope.yaml and copy this code into it:
triggers: - keyword: sensitive_data_scope steps: - say: Setting data... - type: component.sensitive_data.secret_number from: 1 to: 100 - flow_set: secret - user_set: secret - thread_set: secret - say: flow.secret (@ flow.secret ) - say: encrypted_flow.secret (@ encrypted_flow.secret ) - say: redacted_flow.secret (@ redacted_flow.secret ) - say: user.secret (@ user.secret ) - say: encrypted_user.secret (@ encrypted_user.secret ) - say: redacted_user.secret (@ redacted_user.secret ) - say: thread.secret (@ thread.secret ) - say: encrypted_thread.secret (@ encrypted_thread.secret ) - say: redacted_thread.secret (@ redacted_thread.secret ) - say: Done
This flow will use our new
secret_number component to select a number between 1 and 100 and save it to
23 we use the new
redacted_<scope> to access the encrypted and redacted versions of the secret number. Accessing the normal scopes (
<scope>.secret) reveals the unencrypted version.
Push the changes to the Grid by running these commands in your terminal:
meya format meya push
Open the app’s Simulator and enter the keyword
sensitive_data_scope to view the result:
Now let’s take a look at how to use the sensitive data Jinja2 filters to encrypt, decrypt, and redact sensitive data.
The Sensitive Data integration introduces new Jinja2 filters:
Encrypts the data.
Decrypts encrypted sensitive data. Returns an error and stops the flow if it is unable to decrypt the data.
Returns the redacted data (★★★★★).
Let’s add a new flow and test out the sensitive data filters.
In your app’s
flow/sensitive_data folder, create a new file called
filter.yaml and copy this code into it:
triggers: - keyword: sensitive_data_filter steps: - say: Setting data... - type: component.sensitive_data.secret_number from: 1 to: 100 - flow_set: secret: value: (@ encrypted_flow.result ) non_secret: value: (@ flow.result ) invalid: value: 🔐🙈: xyz - say: secret (@ flow.secret.value ) - say: non_secret (@ flow.non_secret.value ) - say: invalid (@ flow.invalid.value ) - say: encrypt_sensitive (@ flow.non_secret.value | encrypt_sensitive ) - say: decrypt_sensitive (@ flow.secret.value | decrypt_sensitive ) - say: try_decrypt_sensitive (@ flow.invalid.value | try_decrypt_sensitive ) - say: redact_sensitive (@ flow.secret.value | redact_sensitive ) - say: Done
Note that we’re reusing the
secret_numbercomponent from the first example, so if you skipped that example, go back and create the component now.
24, we deliberately try to decrypt an invalid payload. This is to simulate what would happen if you tried to decrypt some sensitive data whose TTL had expired. Recall that once the TTL has expired, the data can no longer be decrypted.
Learn more: How long is sensitive data available?
Upload the changes using these commands in your terminal:
meya format meya push
In the Simulator, enter the keyword
sensitive_data_filter to view the result:
With the Sensitive Data integration, you can access new database methods within your components using the
Encrypt an dataclass’s fields that have been flagged
Try to decrypt a dataclass’s fields that have been flagged
Return the redacted version of a dataclass’s fields that have been flagged
Encrypt an entry if its
Try to decrypt an encrypted entry. Return the redacted version as a fallback.
Return the redacted version of an encrypted entry.
Encrypt an object, and specify a custom TTL.
Try to decrypt an encrypted object. Return the redacted version as a fallback.
Decrypt some encrypted data.
If you’ve set up your computer for local Meya development and are using Pycharm as your IDE, you can explore these methods in more detail by taking a look at the Meya SDK code:
When building a custom integration, you may prefer to work with the encrypted or redacted version of
self.entry. You can access them using
One use-case for this is if you want to send an event entry to an analytics integration, but don’t want the sensitive data revealed to the analytics integration. In fact, Meya’s analytics integrations use
self.redacted_entry for this very purpose!
You now know how to use the sensitive data scopes, filters, and Python APIs!
For a more detailed look at how sensitive data works on the Meya platform, check out this guide:
Sensitive data overview
Updated over 1 year ago