The Sensitive Data integration enables:

new data scopes for accessing sensitive data in its encrypted or redacted forms.
new template filters to encrypt, decrypt, and redact sensitive data.
new Python APIs for accessing entries in their encrypted or redacted forms, and methods for encrypting, decrypting, and redacting sensitive data.

Before you start

This guide assumes you’ve set up the Sensitive Data integration. If you haven’t done that yet, check out the set up guide first:

How To Setup the Sensitive Data Integration

Scopes

In this guide, we’ll create a component that generates a random number and flags it as sensitive. We’ll store the number on the data scopes, then test out different ways of viewing the encrypted, decrypted, and redacted versions of the number.

The scopes

The Sensitive Data integration introduces new data scopes:

Data scope	Description
`encrypted_flow`, `encrypted_user`, `encrypted_thread`	Used for accessing encrypted sensitive data stored on `flow`, `user`, and `thread` scopes.
`redacted_flow`, `redacted_user`, `redacted_thread`	Used for accessing redacted (★★★★★) sensitive data on `flow`, `user`, and `thread` scopes.

Create the component

First, let’s create the random number generator component.

In your app’s component folder, create a subfolder called sensitive_data. Inside that folder, create a file called secret_number.py and copy this code into it:

from dataclasses import dataclass
from meya.component.element import Component
from meya.element.field import element_field
from meya.element.field import response_field
from meya.entry import Entry
from random import randint
from typing import List


@dataclass
class SecretNumberComponent(Component):
    @dataclass
    class Response:
        result: int = response_field(sensitive=True)

    sensitive: bool = element_field(default=True)
    from_: int = element_field()
    to: int = element_field()

    async def start(self) -> List[Entry]:
        secret = randint(self.from_, self.to)
        return self.respond(data=self.Response(secret))

This component will generate a random number and store it at flow.result.

📘
Notice that on line 14 we’ve flagged the result property as sensitive. This means that if the Sensitive Data integration is enabled with transcript: true and the entry generated by this component is flagged as sensitive, the result will be available in its encrypted and redacted forms.
In this case, we are forcing the entry to be flagged as sensitive by setting the component’s sensitive property to True by default (line 16).

Create a flow

Now, we’ll create a flow that uses the random number generator and demonstrates what the secret number looks like on the different scopes.

In your app’s flow folder, create a subfolder called sensitive_data. Inside the new folder, create a file called scope.yaml and copy this code into it:

triggers:
  - keyword: sensitive_data_scope

steps:
  - say: Setting data...
  - type: component.sensitive_data.secret_number
    from: 1
    to: 100
  - flow_set: secret
  - user_set: secret
  - thread_set: secret

  - say: flow.secret (@ flow.secret )
  - say: encrypted_flow.secret (@ encrypted_flow.secret )
  - say: redacted_flow.secret (@ redacted_flow.secret )

  - say: user.secret (@ user.secret )
  - say: encrypted_user.secret (@ encrypted_user.secret )
  - say: redacted_user.secret (@ redacted_user.secret )

  - say: thread.secret (@ thread.secret )
  - say: encrypted_thread.secret (@ encrypted_thread.secret )
  - say: redacted_thread.secret (@ redacted_thread.secret )

  - say: Done

This flow will use our new secret_number component to select a number between 1 and 100 and save it to flow, thread, and user scopes.

From lines 13 to 23 we use the new encrypted_<scope> and redacted_<scope> to access the encrypted and redacted versions of the secret number. Accessing the normal scopes (<scope>.secret) reveals the unencrypted version.

Push the changes to the Grid by running these commands in your terminal:

meya format
meya push

Test it out

Open the app’s Simulator and enter the keyword sensitive_data_scope to view the result:

Filters

Now let’s take a look at how to use the sensitive data Jinja2 filters to encrypt, decrypt, and redact sensitive data.

The filters

The Sensitive Data integration introduces new Jinja2 filters:

Filter	Description
`encrypt_sensitive`	Encrypts the data.
`decrypt_sensitive`	Decrypts encrypted sensitive data. Returns an error and stops the flow if it is unable to decrypt the data.
`try_decrypt_sensitive`	Same as `decrypt_sensitive` except it returns the redacted version instead of an error if it is unable to decrypt the data (e.g. the data’s TTL has expired).
`redact_sensitive`	Returns the redacted data (★★★★★).

Create a flow

Let’s add a new flow and test out the sensitive data filters.

In your app’s flow/sensitive_data folder, create a new file called filter.yaml and copy this code into it:

triggers:
  - keyword: sensitive_data_filter

steps:
  - say: Setting data...
  - type: component.sensitive_data.secret_number
    from: 1
    to: 100
  - flow_set:
      secret:
        value: (@ encrypted_flow.result )
      non_secret:
        value: (@ flow.result )
      invalid:
        value:
          🔐🙈: xyz

  - say: secret (@ flow.secret.value )
  - say: non_secret (@ flow.non_secret.value )
  - say: invalid (@ flow.invalid.value )

  - say: encrypt_sensitive (@ flow.non_secret.value | encrypt_sensitive )
  - say: decrypt_sensitive (@ flow.secret.value | decrypt_sensitive )
  - say: try_decrypt_sensitive (@ flow.invalid.value | try_decrypt_sensitive )
  - say: redact_sensitive (@ flow.secret.value | redact_sensitive )

  - say: Done

📘
Note that we’re reusing the secret_number component from the first example, so if you skipped that example, go back and create the component now.

🚧
On line 24, we deliberately try to decrypt an invalid payload. This is to simulate what would happen if you tried to decrypt some sensitive data whose TTL had expired. Recall that once the TTL has expired, the data can no longer be decrypted.
Learn more: How long is sensitive data available?

Upload the changes using these commands in your terminal:

meya format
meya push

Test it out

In the Simulator, enter the keyword sensitive_data_filter to view the result:

Python APIs

With the Sensitive Data integration, you can access new database methods within your components using the self.db object:


`self.db.encrypt_sensitive_fields(sensitive_obj, ttl)`	Encrypt an dataclass’s fields that have been flagged `sensitive=True`, and specify a custom TTL.
`self.db.try_decrypt_sensitive_fields(sensitive_obj)`	Try to decrypt a dataclass’s fields that have been flagged `sensitive=True`. Return the redacted version as a fallback.
`self.db.redact_sensitive_fields(sensitive_obj)`	Return the redacted version of a dataclass’s fields that have been flagged `sensitive=True`.
`self.db.encrypt_sensitive_entry(sensitive_entry, ttl)`	Encrypt an entry if its `sensitive` property is `True`, and specify a custom TTL.
`self.db.try_decrypt_sensitive_entry(sensitive_entry)`	Try to decrypt an encrypted entry. Return the redacted version as a fallback.
`self.db.redact_sensitive_entry(sensitive_entry)`	Return the redacted version of an encrypted entry.
`self.db.encrypt_sensitive(sensitive_obj, ttl)`	Encrypt an object, and specify a custom TTL.
`self.db.try_decrypt_sensitive(ref)`	Try to decrypt an encrypted object. Return the redacted version as a fallback.
`self.db.decrypt_sensitive(ref)`	Decrypt some encrypted data.

📘
If you’ve set up your computer for local Meya development and are using Pycharm as your IDE, you can explore these methods in more detail by taking a look at the Meya SDK code: venv/lib/python3.X/site-packages/meya/db/view/db.py

Entries

When building a custom integration, you may prefer to work with the encrypted or redacted version of self.entry. You can access them using self.encrypted_entry and self.redacted_entry.

One use-case for this is if you want to send an event entry to an analytics integration, but don’t want the sensitive data revealed to the analytics integration. In fact, Meya’s analytics integrations use self.redacted_entry for this very purpose!

👍
You now know how to use the sensitive data scopes, filters, and Python APIs!
For a more detailed look at how sensitive data works on the Meya platform, check out this guide:
Sensitive data overview