Skip to content

Commit

Permalink
Used AWS guidelines safe names in examples
Browse files Browse the repository at this point in the history
  • Loading branch information
seshubaws committed Jan 30, 2024
1 parent cbc2b14 commit a6bb1e3
Show file tree
Hide file tree
Showing 23 changed files with 93 additions and 92 deletions.
2 changes: 1 addition & 1 deletion aws_lambda_powertools/utilities/data_masking/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ def _apply_action_to_fields(
>>> fields = ['a.b.c', 'a.x.y']
# The function will transform the value at 'a.b.c' (1) and 'a.x.y' (2)
# and store the result as:
new_dict = {'a': {'b': {'c': 'transformed_value'}}, 'x': {'y': 'transformed_value'}}
new_dict = {'a': {'b': {'c': '*****'}}, 'x': {'y': '*****'}}
```
"""

Expand Down
7 changes: 5 additions & 2 deletions aws_lambda_powertools/utilities/data_masking/constants.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# The string that replaces values that have been erased
DATA_MASKING_STRING: str = "*****"
# The maximum number of entries that can be retained in the local cryptographic materials cache
CACHE_CAPACITY: int = 100
# The maximum time (in seconds) that a cache entry may be kept in the cache
MAX_CACHE_AGE_SECONDS: float = 300.0
# Maximum number of messages which are allowed to be encrypted under a single cached data key
MAX_MESSAGES_ENCRYPTED: int = 4294967296 # 2 ** 32
# Values can be [1 - 4294967296] (2 ** 32)
MAX_MESSAGES_ENCRYPTED: int = 4294967296
# Maximum number of bytes which are allowed to be encrypted under a single cached data key
MAX_BYTES_ENCRYPTED: int = 9223372036854775807 # 2 ** 63 - 1
# Values can be [1 - 9223372036854775807] (2 ** 63 - 1)
MAX_BYTES_ENCRYPTED: int = 9223372036854775807

ENCRYPTED_DATA_KEY_CTX_KEY = "aws-crypto-public-key"
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def lambda_handler(event, context):
"sensitive": "password"
}
encrypted = data_masker.encrypt(data, fields=["sensitive"])
encrypted = data_masker.encrypt(data)
return encrypted
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def lambda_handler(event, context):
"sensitive": "password"
}
encrypted = data_masker.encrypt(data, fields=["sensitive"])
encrypted = data_masker.encrypt(data)
return encrypted
Expand Down
26 changes: 12 additions & 14 deletions docs/utilities/data_masking.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ stateDiagram-v2

## Terminology

**Erasing** replaces sensitive information **irreversibly** with a non-sensitive placeholder _(`*****`)_. It replaces data in-memory, hence why being irreversible.
**Erasing** replaces sensitive information **irreversibly** with a non-sensitive placeholder _(`*****`)_. This operation replaces data in-memory, making it a one-way action.

**Encrypting** transforms plaintext into ciphertext using an encryption algorithm and a cryptographic key. It allows you to encrypt any sensitive data, so only allowed personnel to decrypt it.

Expand Down Expand Up @@ -99,7 +99,7 @@ Before you start, you will need a KMS symmetric key to encrypt and decrypt your

### Erasing data

Erasing will erase the original data and replace with `*****`. This means you cannot recover erased data, and its type will change to `str`.
Erasing will remove the original data and replace it with a `*****`. This means you cannot recover erased data, and the data type will change to `str` for all data unless the data to be erased is of an Iterable type (`list`, `tuple`, `set`), in which case the method will return a new object of the same type as the input data but with each element replaced by the string `*****`.

=== "getting_started_erase_data.py"
```python hl_lines="4 8 17"
Expand Down Expand Up @@ -134,7 +134,6 @@ Under the hood, we delegate a [number of operations](#encrypt-operation-with-enc
```

1. You can use more than one KMS Key for higher availability but increased latency. </br></br>Encryption SDK will ensure the data key is encrypted with both keys.
2. See [working with nested data](#working-with-nested-data) to learn more about the `fields` parameter. </br></br>If we omit `fields` parameter, the entire dictionary will be encrypted.

=== "generic_data_input.json"
```json hl_lines="7-9 14"
Expand Down Expand Up @@ -165,7 +164,6 @@ Under the hood, we delegate a [number of operations](#decrypt-operation-with-enc

1. Note that KMS key alias or key ID won't work.
2. You can use more than one KMS Key for higher availability but increased latency. </br></br>Encryption SDK will call `Decrypt` API with all master keys when trying to decrypt the data key.
3. See [working with nested data](#working-with-nested-data) to learn more about the `fields` parameter.

=== "encrypt_data_output.json"

Expand Down Expand Up @@ -208,13 +206,12 @@ For a stronger security posture, you can add metadata to each encryption operati

!!! note "We support `JSON` data types only - see [data serialization for more details](#data-serialization-and-preservation)."

You can use the `fields` parameter with dot notation `.` to choose one or more parts of your data to `erase`, `encrypt`, or `decrypt`. This is useful when you want to keep data structure intact except the confidential fields.
You can use the `fields` parameter with dot notation `.` to choose one or more parts of your data to `erase`. This is useful when you want to keep data structure intact except the confidential fields.

When `fields` is present, `erase` and `encrypt` behave differently:
When `fields` is present, `erase` behaves differently:

| Operation | Behavior | Example | Obfuscated |
| --------- | ----------------------------------------------------------- | ----------------------- | ------------------------------- |
| `encrypt` | Obfuscate entire data and replacing with ciphertext string. | `{"cards": ["a", "b"]}` | `{"cards": "ciphertext"}` |
| `erase` | Replace data while keeping collections type intact. | `{"cards": ["a", "b"]}` | `{"cards": ["*****", "*****"]}` |

Here are common scenarios to best visualize how to use `fields`.
Expand Down Expand Up @@ -395,7 +392,8 @@ Note that the return will be a deserialized JSON and your desired fields updated
### Data serialization

???+ note "Current limitations"
1. Python classes, `Dataclasses`, and `Pydantic models` are not supported yet.
1. The `fields` parameter is currently only available to use with the `erase` method, with the potential for it to be added to the `encrypt` and `decrypt` methods in the future.
2. Python classes, `Dataclasses`, and `Pydantic models` are not supported yet.

Before we traverse the data structure, we perform two important operations on input data:

Expand All @@ -418,12 +416,12 @@ For compatibility or performance, you can optionally pass your own JSON serializ

You can modify the following values when initializing the `AWSEncryptionSDKProvider` to best accommodate your security and performance thresholds.

| Parameter | Required | Default | Description |
| -------------------------- | -------- | --------------------- | --------------------------------------------------------------------------------------------- |
| **local_cache_capacity** | | `100` | The maximum number of entries that can be retained in the local cryptographic materials cache |
| **max_cache_age_seconds** | | `300` | The maximum time (in seconds) that a cache entry may be kept in the cache |
| **max_messages_encrypted** | | `4294967296` | The maximum number of messages that may be encrypted under a cache entry |
| **max_bytes_encrypted** | | `9223372036854775807` | The maximum number of bytes that may be encrypted under a cache entry |
| Parameter | Default | Description |
| -------------------------- | --------------------- | --------------------------------------------------------------------------------------------- |
| **local_cache_capacity** | `100` | The maximum number of entries that can be retained in the local cryptographic materials cache |
| **max_cache_age_seconds** | `300` | The maximum time (in seconds) that a cache entry may be kept in the cache |
| **max_messages_encrypted** | `4294967296` | The maximum number of messages that may be encrypted under a cache entry |
| **max_bytes_encrypted** | `9223372036854775807` | The maximum number of bytes that may be encrypted under a cache entry |

**Changing the default algorithm**

Expand Down
12 changes: 6 additions & 6 deletions examples/data_masking/src/choosing_payload_all_nested_keys.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
{
"name": "Leandro",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue",
"postcode": 12345,
"street": "123 Any Street",
"country": "United States",
"timezone": "America/La_Paz"
},
{
"postcode": 94400,
"street": "623 Kraig Mall",
"postcode": 67890,
"street": "100 Main Street",
"country": "United States",
"timezone": "America/Mazatlan"
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "Leandro",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
"*****",
"*****"
Expand Down
16 changes: 8 additions & 8 deletions examples/data_masking/src/choosing_payload_complex_search.json
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Drive"
},
{
"postcode": 91034,
"street": "14987 Avenue 1"
"postcode": 67890,
"street": "111 Main Street"
},
{
"postcode": 78495,
"street": "34452 Avenue 10"
"postcode": 11111,
"street": "100 Any Street"
}
]
}
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"postcode": 12345,
"street": "*****"
},
{
"postcode": 91034,
"postcode": 67890,
"street": "*****"
},
{
"postcode": 78495,
"street": "34452 Avenue 10"
"postcode": 11111,
"street": "100 Any Street"
}
]
}
12 changes: 6 additions & 6 deletions examples/data_masking/src/choosing_payload_list_all_index.json
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Drive"
},
{
"postcode": 91034,
"street": "14987 Avenue 1"
"postcode": 67890,
"street": "100 Main Street,"
}
]
}
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@

{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"postcode": 12345,
"street": "*****"
},
{
"postcode": 91034,
"postcode": 67890,
"street": "*****"
}
]
Expand Down
12 changes: 6 additions & 6 deletions examples/data_masking/src/choosing_payload_list_index.json
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Street"
},
{
"postcode": 91034,
"street": "14987 Avenue 1"
"postcode": 67890,
"street": "100 Main Street"
}
]
}
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@

{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Street"
},
{
"postcode": 91034,
"postcode": 67890,
"street": "*****"
}
]
Expand Down
14 changes: 7 additions & 7 deletions examples/data_masking/src/choosing_payload_list_slice.json
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Street"
},
{
"postcode": 91034,
"street": "14987 Avenue 1"
"postcode": 67890,
"street": "100 Main Street"
},
{
"postcode": 78495,
"street": "34452 Avenue 10"
"street": "111 Any Drive"
}
]
}
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": [
{
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Street"
},
{
"postcode": 91034,
"street": "14987 Avenue 1"
"postcode": 67890,
"street": "100 Main Street"
},
{
"postcode": 78495,
"postcode": 11111,
"street": "*****"
}
]
Expand Down
8 changes: 4 additions & 4 deletions examples/data_masking/src/choosing_payload_multiple_keys.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": {
"postcode": 81847,
"street": "38986 Joanne Stravenue"
"postcode": 12345,
"street": "123 Any Street"
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "Lessa",
"name": "card_number",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": {
"postcode": "*****",
"street": "*****"
Expand Down
6 changes: 3 additions & 3 deletions examples/data_masking/src/choosing_payload_nested_key.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": {
"postcode": 81847
"postcode": 12345
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "Lessa",
"name": "Carlos",
"operation": "non sensitive",
"card_number": "1000 4444 333 2222",
"card_number": "1111 2222 3333 4444",
"address": {
"postcode": "*****"
}
Expand Down
Loading

0 comments on commit a6bb1e3

Please sign in to comment.