# Searchable encryption **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | Searchable encryption enables you to search encrypted records without decrypting the entire database. This is accomplished using *beacons*, which create a map between the plaintext value written to a field and the encrypted value that is actually stored in your database. The AWS Database Encryption SDK stores the beacon in a new field that it adds to the record. Depending on the type of beacon you use, you can perform exact match searches or more customized complex queries on your encrypted data. **Note** Searchable encryption in the AWS Database Encryption SDK differs from the searchable symmetric encryption defined in academic research, such as [searchable symmetric encryption](https://dl.acm.org/doi/10.1145/1180405.1180417). A beacon is a truncated Hash-Based Message Authentication Code (HMAC) tag that creates a map between the plaintext and encrypted values of a field. When you write a new value to an encrypted field that's configured for searchable encryption, the AWS Database Encryption SDK calculates an HMAC over the plaintext value. This HMAC output is a one‐to‐one (1:1) match for the plaintext value of that field. The HMAC output is truncated so that multiple, distinct plaintext values map to the same truncated HMAC tag. These false positives limit an unauthorized user's ability to identify distinguishing information about the plaintext value. When you query a beacon, the AWS Database Encryption SDK automatically filters out these false positives and returns the plaintext result of your query. The average number of false positives generated for each beacon is determined by the beacon length remaining after truncation. For help determining the appropriate beacon length for your implementation, see [Determining beacon length](choosing-beacon-length.md). **Note** Searchable encryption is designed to be implemented in new, unpopulated databases. Any beacon configured in an existing database will only map new records uploaded to the database, there is no way for a beacon to map existing data. **Topics** + [Are beacons right for my dataset?](#are-beacons-right-for-me) + [Searchable encryption scenario](#beacon-overview-example) ## Are beacons right for my dataset? Using beacons to perform queries on encrypted data reduces the performance costs associated with client-side encrypted databases. When you use beacons, there is an inherent tradeoff between how efficient your queries are and how much information is revealed about the distribution of your data. The beacon does not alter the encrypted state of the field. When you encrypt and sign a field with the AWS Database Encryption SDK, the plaintext value of the field is never exposed to the database. The database stores the randomized, encrypted value of the field. Beacons are stored alongside the encrypted fields they are calculated from. This means that even if an unauthorized user cannot view the plaintext values of an encrypted field, they might be able to perform statistical analysis on the beacons to learn more about the distribution of your dataset, and, in extreme cases, identify the plaintext values that a beacon maps to. The way you configure your beacons can mitigate these risks. In particular, [choosing the right beacon length](choosing-beacon-length.md) can help you preserve the confidentiality of your dataset. **Security vs. Performance** + The shorter the beacon length, the more security is preserved. + The longer the beacon length, the more performance is preserved. Searchable encryption might not be able to provide the desired levels of both performance and security for all datasets. Review your threat model, security requirements, and performance needs before configuring any beacons. Consider the following dataset uniqueness requirements as you determine whether searchable encryption is right for your dataset. **Distribution** The amount of security preserved by a beacon depends on the distribution of your dataset. When you configure an encrypted field for searchable encryption, the AWS Database Encryption SDK calculates an HMAC over the plaintext values written to that field. All of the beacons calculated for a given field are calculated using the same key, with the exception of multitenant databases that use a distinct key for each tenant. This means that if the same plaintext value is written to the field multiple times, the same HMAC tag is created for every instance of that plaintext value. You should avoid constructing beacons from fields that contain very common values. For example, consider a database that stores the address of every resident of the state of Illinois. If you construct a beacon from the encrypted `City` field, the beacon calculated over "Chicago" will be overrepresented due to the large percentage of the Illinois population that lives in Chicago. Even if an unauthorized user can only read the encrypted values and beacon values, they might be able to identify which records contain data for residents of Chicago if the beacon preserves this distribution. To minimize the amount of distinguishing information revealed about your distribution, you must sufficiently truncate your beacon. The beacon length required to hide this uneven distribution has significant performance costs that might not meet the needs of your application. You must carefully analyze the distribution of your dataset to determine how much your beacons need to be truncated. The beacon length remaining after truncation directly correlates to the amount of statistical information that can be identified about your distribution. You might need to choose shorter beacon lengths to sufficiently minimize the amount of distinguishing information revealed about your dataset. In extreme cases, you cannot calculate a beacon length for an unevenly distributed dataset that effectively balances performance and security. For example, you should not construct a beacon from a field that stores the result of a medical test for a rare disease. Since `NEGATIVE` results are expected to be significantly more prevalent within the dataset, `POSITIVE` results can be easily identified by how rare they are. It is very challenging to hide the distribution when the field only has two possible values. If you use a beacon length that is short enough to hide the distribution, all plaintext values map to the same HMAC tag. If you use a longer beacon length, it is obvious which beacons map to plaintext `POSITIVE` values. **Correlation** We strongly recommend that you avoid constructing distinct beacons from fields with correlated values. Beacons constructed from correlated fields require shorter beacon lengths to sufficiently minimize the amount of information revealed about the distribution of each dataset to an unauthorized user. You must carefully analyze your dataset, including its entropy and the joint distribution of correlated values, to determine how much your beacons need to be truncated. If the resulting beacon length does not meet your performance needs, then beacons might not be a good fit for your dataset. For example, you should not construct two separate beacons from `City` and `ZIPCode` fields because the ZIP code will likely be associated with just one city. Typically, the false positives generated by a beacon limit an unauthorized user's ability to identify distinguishing information about your dataset. But the correlation between the `City` and `ZIPCode` fields means that an unauthorized user can easily identify which results are false positives and distinguish the different ZIP codes. You should also avoid constructing beacons from fields that contain the same plaintext values. For example, you should not construct a beacon from `mobilePhone` and `preferredPhone` fields because they likely hold the same values. If you construct distinct beacons from both fields, the AWS Database Encryption SDK creates the beacons for each field under different keys. This results in two different HMAC tags for the same plaintext value. The two distinct beacons are unlikely to have the same false positives and an unauthorized user might be able to distinguish different phone numbers. Even if your dataset contains correlated fields or has an uneven distribution, you might be able to construct beacons that preserve the confidentiality of your dataset by using shorter beacon lengths. However, beacon length does not guarantee that every unique value in your dataset will produce a number of false positives that effectively minimizes the amount of distinguishing information revealed about your dataset. Beacon length only estimates the average number of false positives produced. The more unevenly distributed your dataset, the less effective beacon length is at determining the average number of false positives produced. Carefully consider the distribution of the fields you construct beacons from and consider how much you will need to truncate the beacon length to meet your security requirements. The following topics in this chapter assume that your beacons are uniformly distributed and do not contain correlated data. ## Searchable encryption scenario The following example demonstrates a simple searchable encryption solution. In application, the example fields used in this example might not meet the distribution and correlation uniqueness recommendations for beacons. You can use this example for reference as you read about the searchable encryption concepts in this chapter. Consider a database named `Employees` that tracks employee data for a company. Each record in the database contains fields called *EmployeeID*, *LastName*, *FirstName*, and *Address*. Each field in the `Employees` database is identified by the primary key `EmployeeID`. The following is an example plaintext record in the database. ``` { "EmployeeID": 101, "LastName": "Jones", "FirstName": "Mary", "Address": { "Street": "123 Main", "City": "Anytown", "State": "OH", "ZIPCode": 12345 } } ``` If you marked the `LastName` and `FirstName` fields as `ENCRYPT_AND_SIGN` in your [cryptographic actions](concepts.md#crypt-actions), the values in these fields are encrypted locally before they're uploaded to the database. The encrypted data that is uploaded is fully randomized, the database doesn't recognize this data as being protected. It just detects typical data entries. This means that the record that is actually stored in the database might look like the following. ``` { "PersonID": 101, "LastName": "1d76e94a2063578637d51371b363c9682bad926cbd", "FirstName": "21d6d54b0aaabc411e9f9b34b6d53aa4ef3b0a35", "Address": { "Street": "123 Main", "City": "Anytown", "State": "OH", "ZIPCode": 12345 } } ``` If you need to query the database for exact matches in the `LastName` field, [configure a standard beacon](configure-beacons.md#config-standard-beacons) named *LastName* to map the plaintext values written to the `LastName` field to the encrypted values stored in the database. This beacon calculates HMACs from the plaintext values in the `LastName` field. Each HMAC output is truncated so that it is no longer an exact match for the plaintext value. For example, the complete hash and the truncated hash for `Jones` might look like the following. **Complete hash** `2aa4e9b404c68182562b6ec761fcca5306de527826a69468885e59dc36d0c3f824bdd44cab45526f70a2a18322000264f5451acf75f9f817e2b35099d408c833` **Truncated hash** `b35099d408c833` After the standard beacon is configured, you can perform equality searches on the `LastName` field. For example, if you want to search for `Jones`, use the *LastName* beacon to perform the following query. ``` LastName = Jones ``` The AWS Database Encryption SDK automatically filters out the false positives and returns the plaintext result of your query. # Beacons **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | A beacon is a truncated Hash-Based Message Authentication Code (HMAC) tag that creates a map between the plaintext value written to a field and the encrypted value that is actually stored in your database. The beacon does not alter the encrypted state of the field. The beacon calculates an HMAC over the field's plaintext value and stores it alongside the encrypted value. This HMAC output is a one‐to‐one (1:1) match for the plaintext value of that field. The HMAC output is truncated so that multiple, distinct plaintext values map to the same truncated HMAC tag. These false positives limit an unauthorized user's ability to identify distinguishing information about the plaintext value. Beacons can only be constructed from fields that are marked `ENCRYPT_AND_SIGN`, `SIGN_ONLY`, or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` in your [cryptographic actions](concepts.md#crypt-actions). The beacon itself is not signed or encrypted. You cannot construct a beacon with fields that are marked `DO_NOTHING`. The type of beacon you configure determines the type of queries you are able to perform. There are two types of beacons that support searchable encryption. *Standard beacons* perform equality searches. *Compound beacons* combine literal plaintext strings and standard beacons to perform complex database operations. After you [configure your beacons](configure-beacons.md), you must configure a secondary index for each beacon before you can search on the encrypted fields. For more information, see [Configuring secondary indexes with beacons](ddb-searchable-encryption.md#ddb-beacon-indexes). **Topics** + [Standard beacons](#standard-beacon-overview) + [Compound beacons](#compound-beacon-overview) ## Standard beacons Standard beacons are the simplest way to implement searchable encryption in your database. They can only perform equality searches for a single encrypted or virtual field. To learn how to configure standard beacons, see [Configuring standard beacons](configure-beacons.md#config-standard-beacons). The field that a standard beacon is constructed from is called the *beacon source*. It identifies the location of the data that the beacon needs to map. The beacon source can be either an encrypted field or a *virtual field*. The beacon source in each standard beacon must be unique. You cannot configure two beacons with the same beacon source. Standard beacons can be used to perform equality searches for an encrypted or virtual field. Or, they can be used to construct compound beacons to perform more complex database operations. To help you organize and manage standard beacons, the AWS Database Encryption SDK provides the following optional *beacon styles* that define the intended use of a standard beacon. For more information see, [Defining beacon styles](configure-beacons.md#define-beacon-styles). You can create a standard beacon that performs equality searches for a single encrypted field, or you can create a standard beacon that performs equality searches on the concatenation of multiple `ENCRYPT_AND_SIGN`, `SIGN_ONLY`, and `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` fields by creating a virtual field. **Virtual fields** A virtual field is a conceptual field constructed from one or more source fields. Creating a virtual field does not write a new field to your record. The virtual field is not explicitly stored in your database. It is used in standard beacon configuration to give the beacon instructions on how to identify a specific segment of a field or concatenate multiple fields within a record to perform a specific query. A virtual field requires at least one encrypted field. The following example demonstrates the types of transformations and queries you can perform with a virtual field. In application, the example fields used in this example might not meet the [distribution](searchable-encryption.md#searchable-encryption-distribution) and [correlation](searchable-encryption.md#searchable-encryption-correlated-values) uniqueness recommendations for beacons. For example, if you want to perform equality searches on the concatenation of `FirstName` and `LastName` fields, you might create one of the following virtual fields. + A virtual `NameTag` field, constructed from the first letter of the `FirstName` field, followed by the `LastName` field, all in lowercase. This virtual field enables you to query `NameTag=mjones`. + A virtual `LastFirst` field, which is constructed from the `LastName` field, followed by the `FirstName` field. This virtual field enables you to query `LastFirst=JonesMary`. Or, if you want to perform equality searches on a specific segment of an encrypted field, create a virtual field that identifies the segment you want to query. For example, if you want to query an encrypted `IPAddress` field using the first three segments of the IP address, create the following virtual field. + A virtual `IPSegment` field, constructed from `Segments(‘.’, 0, 3)`. This virtual field enables you to query `IPSegment=192.0.2`. The query returns all records with an `IPAddress` value that starts with "192.0.2". Virtual fields must be unique. Two virtual fields cannot be constructed from the exact same source fields. For help configuring virtual fields and the beacons that use them, see [Creating a virtual field](configure-beacons.md#create-virtual-field). ## Compound beacons Compound beacons create indexes that improve query performance and enable you to perform more complex database operations. You can use compound beacons to combine literal plaintext strings and standard beacons to perform complex queries on encrypted records, such as querying two different record types from a single index or querying a combination of fields with a sort key. For more compound beacon solution examples, see [Choose a beacon type](choosing-beacon-type.md). Compound beacons can be constructed from standard beacons or a combination of standard beacons and signed fields. They are constructed from a list of parts. All compound beacons should include a list of [encrypted parts](configure-beacons.md#encrypted-parts) that identifies the `ENCRYPT_AND_SIGN` fields included in the beacon. Every `ENCRYPT_AND_SIGN` field must be identified by a standard beacon. More complex compound beacons might also include a list of [signed parts](configure-beacons.md#signed-parts) that identifies the plaintext `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` fields included in the beacon, and a list of [constructor parts](configure-beacons.md#constructor-parts) that identify all of the possible ways the compound beacon can assemble the fields. **Note** The AWS Database Encryption SDK also supports *signed beacons* that can be configured entirely from plaintext `SIGN_ONLY` and `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` fields. Signed beacons are a type of compound beacon that index and perform complex queries on signed, but not encrypted, fields. For more information, see [Creating signed beacons](configure.md#signed-beacons). For help configuring compound beacons, see [Configuring compound beacons](configure-beacons.md#config-compound-beacons). The way you configure your compound beacon determines the types of queries it can perform. For example, you can make some encrypted and signed parts optional to allow for more flexibility in your queries. For more information on the types of queries compound beacons can perform, see [Querying beacons](using-beacons.md#querying-beacons). # Planning beacons **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | Beacons are designed to be implemented in new, unpopulated databases. Any beacon configured in an existing database will only map new records written to the database. Beacons are calculated from the plaintext value of a field, once the field is encrypted there is no way for the beacon to map existing data. After you have written new records with a beacon, you cannot update the beacon's configuration. However, you can add new beacons for new fields that you add to your record. To implement searchable encryption, you must use the [AWS KMS Hierarchical keyring](use-hierarchical-keyring.md) to generate, encrypt, and decrypt the data keys used to protect your records. For more information, see [Using the Hierarchical keyring for searchable encryption](use-hierarchical-keyring.md#searchable-encryption-hierarchical-keyrings). Before you can configure [beacons](searchable-encryption.md#beacon-definition) for searchable encryption, you need to review your encryption requirements, database access patterns, and threat model to determine the best solution for your database. The [type of beacon](beacons.md) that you configure determines the type of queries that you can perform. The [beacon length](choosing-beacon-length.md) that you specify in standard beacon configuration determines the expected number of false positives produced for a given beacon. We strongly recommend identifying and planning the types of queries that you need to perform before you configure your beacons. Once you have used a beacon, the configuration cannot be updated. We strongly recommend that you review and complete the following tasks before you configure any beacons. + [Determine if beacons are right for your dataset](searchable-encryption.md#are-beacons-right-for-me) + [Choose a beacon type](choosing-beacon-type.md) + [Choose a beacon length](choosing-beacon-length.md) + [Choose a beacon name](choosing-beacon-name.md) Remember the following beacon uniqueness requirements as you plan the searchable encryption solution for your database. + **Every standard beacon must have a unique [beacon source](beacons.md#beacon-source)** Multiple standard beacons cannot be constructed from the same encrypted or virtual field. However, a single standard beacon can be used to construct multiple compound beacons. + **Avoid creating a virtual field with source fields that overlap with existing standard beacons** Constructing a standard beacon from a virtual field that contains a source field that was used to create another standard beacon can reduce the security of both beacons. For more information, see [Security considerations for virtual fields](configure-beacons.md#virtual-field-considerations). ## Considerations for multitenant databases To query beacons configured in a multitenant database, you must include the field that stores the `branch-key-id` associated with the tenant that encrypted the record in your query. You define this field when you [define the beacon key source](use-hierarchical-keyring.md#beacon-key-source). For the query to succeed, the value in this field must identify the appropriate beacon key materials required to recalculate the beacon. Before you configure your beacons, you must decide how you plan to include the `branch-key-id` in your queries. For more information on the different ways you can include the `branch-key-id` in your queries, see [Querying beacons in a multitenant database](searchable-encryption-multitenant.md#query-multitenant-beacons). # Choosing a beacon type **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | With searchable encryption, you can search encrypted records by mapping the plaintext values in an encrypted field with a *beacon*. The type of beacon you configure determines the type of queries that you can perform. We strongly recommend identifying and planning the types of queries that you need to perform before you configure your beacons. After you [configure your beacons](configure-beacons.md), you must configure a secondary index for each beacon before you can search on the encrypted fields. For more information, see [Configuring secondary indexes with beacons](ddb-searchable-encryption.md#ddb-beacon-indexes). Beacons create a map between the plaintext value written to a field and the encrypted value that is actually stored in your database. You cannot compare the values of two standard beacons, even if they contain the same underlying plaintext. The two standard beacons will produce two different HMAC tags for the same plaintext values. As a result, standard beacons cannot perform the following queries. + `beacon1 = beacon2` + `beacon1 IN (beacon2)` + `value IN (beacon1, beacon2, ...)` + `CONTAINS(beacon1, beacon2)` You can only perform the above queries if you compare the [signed parts](configure-beacons.md#signed-parts) of compound beacons, with the exception of the `CONTAINS` operator, which you can use with compound beacons to identify the entire value of an encrypted or signed field that the assembled beacon contains. When you compare signed parts, you can optionally include the prefix of an [encrypted part](configure-beacons.md#encrypted-parts), but you cannot include the encrypted value of a field. For more information about the types of queries that standard and compound beacons can perform, see [Querying beacons](using-beacons.md#querying-beacons). Consider the following searchable encryption solutions as you review your database access patterns. The following examples define which beacon to configure to satisfy different encryption and querying requirements. ## Standard beacons [Standard beacons](beacons.md#standard-beacon-overview) can only perform equality searches. You can use standard beacons to perform the following queries. ### Query a single encrypted field If you want to identify records that contain a specific value for an encrypted field, create a standard beacon. #### Examples For the following example, consider a database named `UnitInspection` that tracks inspection data for a production facility. Each record in the database contains fields called `work_id`, `inspection_date`, `inspector_id_last4`, and `unit`. The full inspector ID is a number between 0—99,999,999. However, to ensure that the dataset is uniformly distributed, the `inspector_id_last4` only stores the last four digits of the inspector's ID. Each field in the database is identified by the primary key `work_id`. The `inspector_id_last4` and `unit` fields are marked `ENCRYPT_AND_SIGN` in the [cryptographic actions](concepts.md#crypt-actions). The following is an example of a plaintext entry in the `UnitInspection` database. ``` { "work_id": "1c7fcff3-6e74-41a8-b7f7-925dc039830b", "inspection_date": 2023-06-07, "inspector_id_last4": 8744, "unit": 229304973450 } ``` **Query a single encrypted field in a record** If the `inspector_id_last4` field needs to be encrypted, but you still need query it for exact matches, construct a standard beacon from the `inspector_id_last4` field. Then, use the standard beacon to create a secondary index. You can use this secondary index to query on the encrypted `inspector_id_last4` field. For help configuring standard beacons, see [Configuring standard beacons](configure-beacons.md#config-standard-beacons). ### Query a virtual field A [virtual field](beacons.md#virtual-field) is a conceptual field constructed from one or more source fields. If you want to perform equality searches for a specific segment of an encrypted field, or perform equality searches on the concatenation of multiple fields, construct a standard beacon from a virtual field. All virtual fields must include at least one encrypted source field. #### Examples The following examples create virtual fields for the `Employees` database. The following is an example plaintext record in the `Employees` database. ``` { "EmployeeID": 101, "SSN": 000-00-0000, "LastName": "Jones", "FirstName": "Mary", "Address": { "Street": "123 Main", "City": "Anytown", "State": "OH", "ZIPCode": 12345 } } ``` **Query a segment of an encrypted field** For this example, the `SSN` field is encrypted. If you want to query the `SSN` field using the last four digits of a social security number, create a virtual field that identifies the segment you plan to query. A virtual `Last4SSN` field, constructed from `Suffix(4)` enables you to query `Last4SSN=0000`. Use this virtual field to construct a standard beacon. Then, use the standard beacon to create a secondary index. You can use this secondary index to query on the virtual field. This query returns all records with an `SSN` value that ends with the last four digits you specified. **Query the concatenation of multiple fields** The following example demonstrates the types of transformations and queries you can perform with a virtual field. In application, the example fields used in this example might not meet the [distribution](searchable-encryption.md#searchable-encryption-distribution) and [correlation](searchable-encryption.md#searchable-encryption-correlated-values) uniqueness recommendations for beacons. If you want to perform equality searches on a concatenation of `FirstName` and `LastName` fields, you might create a virtual `NameTag` field, constructed from the first letter of the `FirstName` field, followed by the `LastName` field, all in lowercase. Use this virtual field to construct a standard beacon. Then, use the standard beacon to create a secondary index. You can use this secondary index to query `NameTag=mjones` on the virtual field. At least one of the source fields must be encrypted. Either `FirstName` or `LastName` could be encrypted, or they could both be encrypted. Any plaintext source fields must be marked as `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` in your [cryptographic actions](concepts.md#crypt-actions). For help configuring virtual fields and the beacons that use them, see [Creating a virtual field](configure-beacons.md#create-virtual-field). ## Compound beacons [Compound beacons](beacons.md#compound-beacon-overview) create an index from literal plaintext strings and standard beacons to perform complex database operations. You can use compound beacons to perform the following queries. ### Query a combination of encrypted fields on a single index If you need to query a combination of encrypted fields on a single index, create a compound beacon that combines the individual standard beacons constructed for each encrypted field to form a single index. After you configure the compound beacon, you can create a secondary index that specifies the compound beacon as the partition key to perform exact match queries or with a sort key to perform more complex queries. Secondary indexes that specify the compound beacon as the sort key can perform exact match queries and more customized complex queries. #### Examples For the following examples, consider a database named `UnitInspection` that tracks inspection data for a production facility. Each record in the database contains fields called `work_id`, `inspection_date`, `inspector_id_last4`, and `unit`. The full inspector ID is a number between 0—99,999,999. However, to ensure that the dataset is uniformly distributed, the `inspector_id_last4` only stores the last four digits of the inspector's ID. Each field in the database is identified by the primary key `work_id`. The `inspector_id_last4` and `unit` fields are marked `ENCRYPT_AND_SIGN` in the [cryptographic actions](concepts.md#crypt-actions). The following is an example of a plaintext entry in the `UnitInspection` database. ``` { "work_id": "1c7fcff3-6e74-41a8-b7f7-925dc039830b", "inspection_date": 2023-06-07, "inspector_id_last4": 8744, "unit": 229304973450 } ``` **Perform equality searches on a combination of encrypted fields** If you want to query the `UnitInspection` database for exact matches on `inspector_id_last4.unit`, first create distinct standard beacons for the `inspector_id_last4` and `unit` fields. Then, create a compound beacon from the two standard beacons. After you configure the compound beacon, create a secondary index that specifies the compound beacon as the partition key. Use this secondary index to query for exact matches on `inspector_id_last4.unit`. For example, you could query this beacon to find a list of inspections that an inspector performed for a given unit. **Perform complex queries on a combination of encrypted fields** If you want to query the `UnitInspection` database on `inspector_id_last4` and `inspector_id_last4.unit`, first create distinct standard beacons for the `inspector_id_last4` and `unit` fields. Then, create a compound beacon from the two standard beacons. After you configure the compound beacon, create a secondary index that specifies the compound beacon as the sort key. Use this secondary index to query the `UnitInspection` database for entries that start with a certain inspector or query the database for a list of all of the units within a specific unit ID range that were inspected by a certain inspector. You can also perform exact match searches on `inspector_id_last4.unit`. For help configuring compound beacons, see [Configuring compound beacons](configure-beacons.md#config-compound-beacons). ### Query a combination of encrypted and plaintext fields on a single index If you need to query a combination of encrypted and plaintext fields on a single index, create a compound beacon that combines individual standard beacons and plaintext fields to form a single index. The plaintext fields used to construct the compound beacon must be marked `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` in your [cryptographic actions](concepts.md#crypt-actions). After you configure the compound beacon, you can create a secondary index that specifies the compound beacon as the partition key to perform exact match queries or with a sort key to perform more complex queries. Secondary indexes that specify the compound beacon as the sort key can perform exact match queries and more customized complex queries. #### Examples For the following examples, consider a database named `UnitInspection` that tracks inspection data for a production facility. Each record in the database contains fields called `work_id`, `inspection_date`, `inspector_id_last4`, and `unit`. The full inspector ID is a number between 0—99,999,999. However, to ensure that the dataset is uniformly distributed, the `inspector_id_last4` only stores the last four digits of the inspector's ID. Each field in the database is identified by the primary key `work_id`. The `inspector_id_last4` and `unit` fields are marked `ENCRYPT_AND_SIGN` in the [cryptographic actions](concepts.md#crypt-actions). The following is an example of a plaintext entry in the `UnitInspection` database. ``` { "work_id": "1c7fcff3-6e74-41a8-b7f7-925dc039830b", "inspection_date": 2023-06-07, "inspector_id_last4": 8744, "unit": 229304973450 } ``` **Perform equality searches on a combination of fields** If you want to query the `UnitInspection` database for inspections conducted by a specific inspector on a specific date, first create a standard beacon for the `inspector_id_last4` field. The `inspector_id_last4` field is marked `ENCRYPT_AND_SIGN` in the [cryptographic actions](concepts.md#crypt-actions). All encrypted parts require their own standard beacon. The `inspection_date` field is marked `SIGN_ONLY` and does not require a standard beacon. Next, create a compound beacon from the `inspection_date` field and the `inspector_id_last4` standard beacon. After you configure the compound beacon, create a secondary index that specifies the compound beacon as the partition key. Use this secondary index to query the databases for records with exact matches to a certain inspector and inspection date. For example, you can query the database for a list of all inspections that the inspector whose ID ends with `8744` conducted on a specific date. **Perform complex queries on a combination of fields** If you want to query the database for inspections conducted within an `inspection_date` range, or query the database for inspections conducted on a particular `inspection_date` constrained by `inspector_id_last4` or `inspector_id_last4.unit`, first create distinct standard beacons for the `inspector_id_last4` and `unit` fields. Then, create a compound beacon from the plaintext `inspection_date` field and the two standard beacons. After you configure the compound beacon, create a secondary index that specifies the compound beacon as the sort key. Use this secondary index to perform queries for inspections conducted on specific dates by a specific inspector. For example, you can query the database for a list of all units inspected on the same date. Or, you can query the database for a list of all inspections performed on a specific unit between a given range of inspection dates. For help configuring compound beacons, see [Configuring compound beacons](configure-beacons.md#config-compound-beacons). # Choosing a beacon length **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | When you write a new value to an encrypted field that's configured for searchable encryption, the AWS Database Encryption SDK calculates an HMAC over the plaintext value. This HMAC output is a one‐to‐one (1:1) match for the plaintext value of that field. The HMAC output is truncated so that multiple, distinct plaintext values map to the same truncated HMAC tag. These collisions, or *false positives*, limit an unauthorized user's ability to identify distinguishing information about the plaintext value. The average number of false positives generated for each beacon is determined by the beacon length remaining after truncation. You only need to define beacon length when configuring standard beacons. Compound beacons use the beacon lengths of the standard beacons they're constructed from. The beacon does not alter the encrypted state of the field. However, when you use beacons, there is an inherent tradeoff between how efficient your queries are and how much information is revealed about the distribution of your data. The goal of searchable encryption is to reduce the performance costs associated with client-side encrypted databases by using beacons to perform queries on encrypted data. Beacons are stored alongside the encrypted fields they are calculated from. This means that they can reveal distinguishing information about the distribution of your dataset. In extreme cases, an unauthorized user might be able to analyze the information revealed about your distribution and use it to identify a field's plaintext value. Choosing the right beacon length can help mitigate these risks and preserve the confidentiality of your distribution. Review your threat model to determine the level of security that you need. For example, the more individuals who have access to your database, but should not have access to the plaintext data, the more you might want to protect the confidentiality of your dataset distribution. To increase confidentiality, a beacon needs to generate more false positives. Increased confidentiality results in reduced query performance. **Security vs. Performance** + A beacon length that is **too long** produces too few false positives and might reveal distinguishing information about the distribution of your dataset. + A beacon length that is **too short** produces too many false positives and increases the performance cost of queries because it requires a broader scan of the database. When determining the appropriate beacon length for your solution, you must find a length that adequately preserves the security of your data without impacting the performance of your queries more than absolutely necessary. The amount of security preserved by a beacon depends on the [distribution](searchable-encryption.md#searchable-encryption-distribution) of your dataset and the [correlation](searchable-encryption.md#searchable-encryption-correlated-values) of the fields that your beacons are constructed from. The following topics assume that your beacons are uniformly distributed and do not contain correlated data. **Topics** + [Calculating beacon length](#calculate-beacon-length) + [Example](#beacon-length-example) ## Calculating beacon length Beacon length is defined in *bits* and refers to the number of bits of the HMAC tag that are kept after truncation. The recommended beacon length varies depending on the dataset distribution, presence of correlated values, and your specific security and performance requirements. If your dataset is uniformly distributed, you can use the following equations and procedures to help identify the best beacon length for your implementation. These equations only estimate the average number of false positives that the beacon will produce, they do not guarantee that every unique value in your dataset will produce a specific number of false positives. **Note** The effectiveness of these equations is dependent on the distribution of your dataset. If your dataset is not uniformly distributed, see [Are beacons right for my dataset?](searchable-encryption.md#are-beacons-right-for-me). In general, the further your dataset is from a uniform distribution, the more you need to shorten your beacon length. 1. **Estimate the population** The population is the expected number of unique values in the field that your standard beacon is constructed from, it is not the total expected number of values stored in the field. For example, consider an encrypted `Room` field that identifies the location of employee meetings. The `Room` field is expected to store 100,000 total values, but there are only 50 different rooms that employees can reserve for meetings. This means that the population is 50 because there only 50 possible unique values that can be stored in the `Room` field. **Note** If your standard beacon is constructed from a [virtual field](beacons.md#virtual-field), the population used to calculate beacon length is the number of unique combinations created by the virtual field. When estimating your population, be sure to consider the projected growth of the dataset. After you have written new records with the beacon, you cannot update the beacon length. Review your threat model and any existing database solutions to create an estimate for the number of unique values you expect this field to store in the next five years. Your population does not need to be precise. First, identify the number of unique values in your current database, or estimate the number of unique values that you expect to store in the first year. Next, use the following questions to help you determine the projected growth of unique values over the next five years. + Do you expect the unique values to multiply by 10? + Do you expect the unique values to multiply by 100? + Do you expect the unique values to multiply by 1000? The difference between 50,000 and 60,000 unique values is not significant and they will both result in the same recommended beacon length. However, the difference between 50,000 and 500,000 unique values will significantly impact the recommended beacon length. Consider reviewing public data on the frequency of common data types, such as ZIP codes or last names. For example, there are 41,707 ZIP codes in the United States. The population you use should be proportional to your own database. If the `ZIPCode` field in your database includes data from across the entire United States, then you might define your population as 41,707, even if the `ZIPCode` field does not *currently* have 41,707 unique values. If the `ZIPCode` field in your database only includes data from a single state, and will only ever include data from a single state, then you might define your population as the total number of ZIP codes in that state instead of 41,704. 1. **Calculate the recommended range for the expected number of collisions** To determine the appropriate beacon length for a given field, you must first identify an appropriate range for the expected number of collisions. The expected number of collisions represents the average expected number of unique plaintext values that map to a particular HMAC tag. The expected number of false positives for one unique plaintext value is one less than the expected number of collisions. We recommend that the expected number of collisions is greater than or equal to two, and less than the square root of your population. The following equations only work if your population has 16 or more unique values. ``` 2 ≤ number of collisions < √(Population) ``` If the number of collisions is less than two, the beacon will produce too few false positives. We recommend two as the minimum number of expected collisions because it means, on average, every unique value in the field will generate at least one false positive by mapping to one other unique value. 1. **Calculate the recommended range for beacon lengths** After identifying the minimum and maximum number of expected collisions, use the following equation to identify a range of appropriate beacon lengths. ``` number of collisions = Population * 2-(beacon length) ``` First, solve for **beacon length** where the number of expected collisions equals two (the minimum recommended number of expected collisions). ``` 2 = Population * 2-(beacon length) ``` Then, solve for **beacon length** where the expected number of collisions equals the square root of your population (the maximum recommended number of expected collisions). ``` √(Population) = Population * 2-(beacon length) ``` We recommend rounding the output produced by this equation down to the shorter beacon length. For example, if the equation produces a beacon length of 15.6, we recommend rounding that value down to 15 bits instead of rounding up to 16 bits. 1. **Choose a beacon length** These equations only identify a recommended range of beacon lengths for your field. We recommend using a shorter beacon length to preserve the security of your dataset whenever possible. However, the beacon length that you actually use is determined by your threat model. Consider your performance requirements as you review your threat model to determine the best beacon length for your field. Using a shorter beacon length reduces query performance, while using a longer beacon length decreases security. In general, if your dataset is unevenly [distributed](searchable-encryption.md#searchable-encryption-distribution), or if you construct distinct beacons from [correlated](searchable-encryption.md#searchable-encryption-correlated-values) fields, you need to use shorter beacon lengths to minimize the amount of information revealed about the distribution of your datasets. If you review your threat model and decide that any distinguishing information revealed about the distribution of a field does not present a threat to your overall security, you might choose to use a beacon length that is longer than the recommended range you calculated. For example, if you calculated the recommended range of beacon lengths for a field as 9—16 bits, you might choose to use a beacon length of 24 bits to avoid any performance loss. Choose your beacon length carefully. After you have written new records with the beacon, you cannot update the beacon length. ## Example Consider a database that marked the `unit` field as `ENCRYPT_AND_SIGN` in the [cryptographic actions](concepts.md#crypt-actions). To configure a standard beacon for the `unit` field, we need to determine the expected number of false positives and beacon length for the `unit` field. 1. Estimate the population After reviewing our threat model and current database solution, we expect the `unit` field to eventually have 100,000 unique values. This means that **Population = 100,000**. 1. Calculate the recommended range for the expected number of collisions. For this example, the expected number of collisions should be between 2—316. ``` 2 ≤ number of collisions < √(Population) ``` 1. ``` 2 ≤ number of collisions < √(100,000) ``` 1. ``` 2 ≤ number of collisions < 316 ``` 1. Calculate the recommended range for beacon length. For this example, the beacon length should be between 9—16 bits. ``` number of collisions = Population * 2-(beacon length) ``` 1. Calculate the beacon length where the expected number of collisions equals the minimum identified in **Step 2**. ``` 2 = 100,000 * 2-(beacon length) ``` Beacon length = 15.6, or 15 bits 1. Calculate the beacon length where the expected number of collisions equals the maximum identified in **Step 2**. ``` 316 = 100,000 * 2-(beacon length) ``` Beacon length = 8.3, or 8 bits 1. Determine the beacon length appropriate for your security and performance requirements. For every bit below 15, the performance cost and the security double. + 16 bits + On average, each unique value will map to 1.5 other units. + Security: two records with the same truncated HMAC tag are 66% likely to have the same plaintext value. + Performance: a query will retrieve 15 records for every 10 records that you actually requested. + 14 bits + On average, each unique value will map to 6.1 other units. + Security: two records with the same truncated HMAC tag are 33% likely to have the same plaintext value. + Performance: a query will retrieve 30 records for every 10 records that you actually requested. # Choosing a beacon name **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | Every beacon is identified by a unique *beacon name*. Once a beacon is configured, the beacon name is the name you use when querying an encrypted field. A beacon name can be the same name as an encrypted field or [virtual field](beacons.md#virtual-field), but it cannot be the same name as an unencrypted field. Two different beacons cannot have the same beacon name. For examples demonstrating how to name and configure beacons, see [Configuring beacons](configure-beacons.md). **Naming standard beacon** When naming standard beacons, we strongly recommend that your beacon name resolves to the [*beacon source*](beacons.md#beacon-source) whenever possible. This means that the beacon name and the name of the encrypted or [virtual](beacons.md#virtual-field) field that your standard beacon is constructed from are the same. For example, if you are creating a standard beacon for an encrypted field named `LastName`, your beacon name should also be `LastName`. When your beacon name is the same as the beacon source, you can omit the beacon source from your configuration and the AWS Database Encryption SDK will automatically use the beacon name as the beacon source. # Configuring beacons **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | There are two types of beacons that support searchable encryption. Standard beacons perform equality searches. They are the simplest way to implement searchable encryption in your database. Compound beacons combine literal plaintext strings and standard beacons to perform more complex queries. Beacons are designed to be implemented in new, unpopulated databases. Any beacon configured in an existing database will only map new records written to the database. Beacons are calculated from the plaintext value of a field, once the field is encrypted there is no way for the beacon to map existing data. After you have written new records with a beacon, you cannot update the beacon's configuration. However, you can add new beacons for new fields that you add to your record. After determining your access patterns, configuring beacons should be the second step in your database implementation. Then, after you configure all of your beacons, you need to create an [AWS KMS Hierarchical keyring](use-hierarchical-keyring.md), define the beacon version, [configure a secondary index for each beacon](ddb-searchable-encryption.md#ddb-beacon-indexes), define your [cryptographic actions](concepts.md#crypt-actions), and configure your database and AWS Database Encryption SDK client. For more information, see [Using beacons](using-beacons.md). To make it easier to define the beacon version, we recommend creating lists for standard and compound beacons. Add each beacon you create to the respective standard or compound beacon list as you configure them. **Topics** + [Configuring standard beacons](#config-standard-beacons) + [Configuring compound beacons](#config-compound-beacons) + [Example configurations](beacon-config-examples.md) ## Configuring standard beacons [Standard beacons](beacons.md#standard-beacon-overview) are the simplest way to implement searchable encryption in your database. They can only perform equality searches for a single encrypted or virtual field. ### Example configuration syntax ------ #### [ Java ] ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("beaconName") .length(beaconLengthInBits) .build(); standardBeaconList.add(exampleStandardBeacon); ``` ------ #### [ C\$1 / .NET ] ``` var standardBeaconList = new List(); StandardBeacon exampleStandardBeacon = new StandardBeacon { Name = "beaconName", Length = 10 }; standardBeaconList.Add(exampleStandardBeacon); ``` ------ #### [ Rust ] ``` let standard_beacon_list = vec![ StandardBeacon::builder().name("beacon_name").length(beacon_length_in_bits).build()?, ``` ------ To configure a standard beacon, provide the following values. **Beacon name** The name you use when querying an encrypted field. A beacon name can be the same name as an encrypted field or virtual field, but it cannot be the same name as an unencrypted field. We strongly recommend using the name of the encrypted field or [virtual field](beacons.md#virtual-field) that your standard beacon is constructed from whenever possible. Two different beacons cannot have the same beacon name. For help determining the best beacon name for your implementation, see [Choosing a beacon name](choosing-beacon-name.md). **Beacon length** The number of bits of the beacon hash value that are kept after truncation. The beacon length determines the average number of false positives produced by a given beacon. For more information and help determining the appropriate beacon length for your implementation, see [Determining beacon length](choosing-beacon-length.md). **Beacon source (Optional)** The field that a standard beacon is constructed from. The beacon source must be a field name or an index referring to the value of a nested field. When your beacon name is the same as the beacon source, you can omit the the beacon source from your configuration and the AWS Database Encryption SDK will automatically use the beacon name as the beacon source. ### Creating a virtual field To create a [virtual field](beacons.md#virtual-field), you must provide a name for the virtual field and a list of the source fields. The order that you add source fields to the virtual part list determines the order that they are concatenated to build the virtual field. The following example concatenates two source fields in their entirety to create a virtual field. **Note** We recommend verifying that your virtual fields produce the expected outcome before you populate your database. For more information, see [Testing beacon outputs](ddb-searchable-encryption.md#ddb-beacon-testing). ------ #### [ Java ] **See the complete code example**: [VirtualBeaconSearchableEncryptionExample.java](https://github.com/aws/aws-database-encryption-sdk-dynamodb//blob/main/Examples/runtimes/java/DynamoDbEncryption/src/main/java/software/amazon/cryptography/examples/searchableencryption/VirtualBeaconSearchableEncryptionExample.java) ``` List virtualPartList = new ArrayList<>(); virtualPartList.add(sourceField1); virtualPartList.add(sourceField2); VirtualField virtualFieldName = VirtualField.builder() .name("virtualFieldName") .parts(virtualPartList) .build(); List virtualFieldList = new ArrayList<>(); virtualFieldList.add(virtualFieldName); ``` ------ #### [ C\$1 / .NET ] **See the complete code example**: [VirtualBeaconSearchableEncryptionExample.cs](https://github.com/aws/aws-database-encryption-sdk-dynamodb/tree/main/Examples/runtimes/net/src/searchableencryption/VirtualBeaconSearchableEncryptionExample.cs) ``` var virtualPartList = new List { sourceField1, sourceField2 }; var virtualFieldName = new VirtualField { Name = "virtualFieldName", Parts = virtualPartList }; var virtualFieldList = new List { virtualFieldName }; ``` ------ #### [ Rust ] **See the complete code example**: [virtual\$1beacon\$1searchable\$1encryption.rs](https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/main/releases/rust/db_esdk/examples/searchableencryption/virtual_beacon_searchable_encryption.rs) ``` let virtual_part_list = vec![source_field_one, source_field_two]; let state_and_has_test_result_field = VirtualField::builder() .name("virtual_field_name") .parts(virtual_part_list) .build()?; let virtual_field_list = vec![virtual_field_name]; ``` ------ To create a virtual field with a specific segment of a source field, you must define that transformation before adding the source field to your virtual part list. #### Security considerations for virtual fields Beacons do not alter the encrypted state of the field. However, when you use beacons, there is an inherent tradeoff between how efficient your queries are and how much information is revealed about the distribution of your data. The way that you configure your beacon determines the level of security that is preserved by that beacon. Avoid creating a virtual field with source fields that overlap with existing standard beacons. Creating virtual fields that include a source field that has already been used to create a standard beacon can reduce the level of security for both beacons. The extent that security is reduced is dependent on the level of entropy added by the additional source fields. The level of entropy is determined by the distribution of unique values in the additional source field and the number of bits that the additional source field contributes to the overall size of the virtual field. You can use population and [beacon length](choosing-beacon-length.md) to determine if the source fields for a virtual field preserve the security of your dataset. The population is the expected number of unique values in a field. Your population does not need to be precise. For help estimating the population of a field, see [Estimate the population](choosing-beacon-length.md#estimate-population). Consider the following example as you review the security of your virtual fields. + Beacon1 is constructed from `FieldA`. `FieldA` has a population greater than **2(Beacon1 length)**. + Beacon2 is constructed from `VirtualField`, which is constructed from `FieldA`, `FieldB`, `FieldC`, and `FieldD`. Together, `FieldB`, `FieldC`, and `FieldD` have a population greater than **2N** Beacon2 preserves the security of both Beacon1 and Beacon2 if the following statements are true: ``` N ≥ (Beacon1 length)/2 ``` and ``` N ≥ (Beacon2 length)/2 ``` ### Defining beacon styles Standard beacons can be used to perform equality searches for an encrypted or virtual field. Or, they can be used to construct compound beacons to perform more complex database operations. To help you organize and manage standard beacons, the AWS Database Encryption SDK provides the following optional *beacon styles* that define the intended use of a standard beacon. **Note** To define beacon styles, you must use version 3.2 or later of the AWS Database Encryption SDK. Deploy the new version to all readers before adding beacon styles to your beacon configurations. ------ #### [ PartOnly ] A standard beacon defined as `PartOnly` can only be used to define an [encrypted part](#encrypted-parts) of a compound beacon. You cannot directly query a `PartOnly` standard beacon. **Java** ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("beaconName") .length(beaconLengthInBits) .style( BeaconStyle.builder() .partOnly(PartOnly.builder().build()) .build() ) .build(); standardBeaconList.add(exampleStandardBeacon); ``` **C\$1 / .NET** ``` new StandardBeacon { Name = "beaconName", Length = beaconLengthInBits, Style = new BeaconStyle { PartOnly = new PartOnly() } } ``` **Rust** ``` StandardBeacon::builder() .name("beacon_name") .length(beacon_length_in_bits) .style(BeaconStyle::PartOnly(PartOnly::builder().build()?)) .build()? ``` ------ #### [ Shared ] By default, every standard beacon generates a unique HMAC key for beacon calculation. As a result, you cannot perform an equality search on the encrypted fields from two separate standard beacons. A standard beacon defined as `Shared` uses the HMAC key from another standard beacon for its calculations. For example, if you need to compare `beacon1` fields to `beacon2` fields, define `beacon2` as a `Shared` beacon that uses the HMAC key from `beacon1` for its calculations. **Note** Consider your security and performance needs before configuring any `Shared` beacons. `Shared` beacons might increase the amount of statistical information that can be identified about the distribution of your dataset. For example, they might reveal which shared fields contain the same plaintext value. **Java** ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("beacon2") .length(beaconLengthInBits) .style( BeaconStyle.builder() .shared(Shared.builder().other("beacon1").build()) .build() ) .build(); standardBeaconList.add(exampleStandardBeacon); ``` **C\$1 / .NET** ``` new StandardBeacon { Name = "beacon2", Length = beaconLengthInBits, Style = new BeaconStyle { Shared = new Shared { Other = "beacon1" } } } ``` **Rust** ``` StandardBeacon::builder() .name("beacon2") .length(beacon_length_in_bits) .style(BeaconStyle::Shared( Shared::builder().other("beacon1").build()?, )) .build()? ``` ------ #### [ AsSet ] By default, if a field value is a set, the AWS Database Encryption SDK calculates a single standard beacon for the set. As a result, you cannot perform the query `CONTAINS(a, :value)` where `a` is an encrypted field. A standard beacon defined as `AsSet` calculates individual standard beacon values for each individual element of the set and stores the beacon value in the item as a set. This enables the AWS Database Encryption SDK to perform the query `CONTAINS(a, :value)`. To define an `AsSet` standard beacon, the elements in the set must be from the same population so that they can all use the same [beacon length](choosing-beacon-length.md). The beacon set might have fewer elements than the plaintext set if there were collisions when calculating the beacon values. **Note** Consider your security and performance needs before configuring any `AsSet` beacons. `AsSet` beacons might increase the amount of statistical information that can be identified about the distribution of your dataset. For example, they might reveal the size of the plaintext set. **Java** ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("beaconName") .length(beaconLengthInBits) .style( BeaconStyle.builder() .asSet(AsSet.builder().build()) .build() ) .build(); standardBeaconList.add(exampleStandardBeacon); ``` **C\$1 / .NET** ``` new StandardBeacon { Name = "beaconName", Length = beaconLengthInBits, Style = new BeaconStyle { AsSet = new AsSet() } } ``` **Rust** ``` StandardBeacon::builder() .name("beacon_name") .length(beacon_length_in_bits) .style(BeaconStyle::AsSet(AsSet::builder().build()?)) .build()? ``` ------ #### [ SharedSet ] A standard beacon defined as `SharedSet` combines the `Shared` and `AsSet` functions so that you can perform equality searches on the encrypted values of a set and field. This enables the AWS Database Encryption SDK to perform the query `CONTAINS(a, b)` where `a` is an encrypted set and `b` is an encrypted field. **Note** Consider your security and performance needs before configuring any `Shared` beacons. `SharedSet` beacons might increase the amount of statistical information that can be identified about the distribution of your dataset. For example, they might reveal the size of the plaintext set or which shared fields contain the same plaintext value. **Java** ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("beacon2") .length(beaconLengthInBits) .style( BeaconStyle.builder() .sharedSet(SharedSet.builder().other("beacon1").build()) .build() ) .build(); standardBeaconList.add(exampleStandardBeacon); ``` **C\$1 / .NET** ``` new StandardBeacon { Name = "beacon2", Length = beaconLengthInBits, Style = new BeaconStyle { SharedSet = new SharedSet { Other = "beacon1" } } } ``` **Rust** ``` StandardBeacon::builder() .name("beacon2") .length(beacon_length_in_bits) .style(BeaconStyle::SharedSet( SharedSet::builder().other("beacon1").build()?, )) .build()? ``` ------ ## Configuring compound beacons Compound beacons combine literal plaintext strings and standard beacons to perform complex database operations, such as querying two different record types from a single index or querying a combination of fields with a sort key. Compound beacons can be constructed from `ENCRYPT_AND_SIGN`, `SIGN_ONLY`, and `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` fields. You must create a standard beacon for every encrypted field included in the compound beacon. **Note** We recommend verifying that your compound beacons produce the expected outcome before you populate your database. For more information, see [Testing beacon outputs](ddb-searchable-encryption.md#ddb-beacon-testing). ### Example configuration syntax ------ #### [ Java ] **Compound beacon configuration** The following example defines encrypted and signed parts lists locally within the compound beacon configuration. ``` List compoundBeaconList = new ArrayList<>(); CompoundBeacon exampleCompoundBeacon = CompoundBeacon.builder() .name("compoundBeaconName") .split(".") .encrypted(encryptedPartList) .signed(signedPartList) .constructors(constructorList) .build(); compoundBeaconList.add(exampleCompoundBeacon); ``` **Beacon version definition** The following example defines encrypted and signed parts lists globally in the beacon version. For more information on defining the beacon version, see [Using beacons](using-beacons.md). ``` List beaconVersions = new ArrayList<>(); beaconVersions.add( BeaconVersion.builder() .standardBeacons(standardBeaconList) .compoundBeacons(compoundBeaconList) .encryptedParts(encryptedPartList) .signedParts(signedPartList) .version(1) // MUST be 1 .keyStore(keyStore) .keySource(BeaconKeySource.builder() .single(SingleKeyStore.builder() .keyId(branchKeyId) .cacheTTL(6000) .build()) .build()) .build() ); ``` ------ #### [ C\$1 / .NET ] **See the complete code sample**: [BeaconConfig.cs](https://github.com/aws/aws-database-encryption-sdk-dynamodb/tree/main/Examples/runtimes/net/src/searchableencryption/complexexample/BeaconConfig.cs) **Compound beacon configuration** The following example defines encrypted and signed parts lists locally within the compound beacon configuration. ``` var compoundBeaconList = new List(); var exampleCompoundBeacon = new CompoundBeacon { Name = "compoundBeaconName", Split = ".", Encrypted = encryptedPartList, Signed = signedPartList, Constructors = constructorList }; compoundBeaconList.Add(exampleCompoundBeacon); ``` **Beacon version definition** The following example defines encrypted and signed parts lists globally in the beacon version. For more information on defining the beacon version, see [Using beacons](using-beacons.md). ``` var beaconVersions = new List { new BeaconVersion { StandardBeacons = standardBeaconList, CompoundBeacons = compoundBeaconList, EncryptedParts = encryptedPartsList, SignedParts = signedPartsList, Version = 1, // MUST be 1 KeyStore = keyStore, KeySource = new BeaconKeySource { Single = new SingleKeyStore { KeyId = branchKeyId, CacheTTL = 6000 } } } }; ``` ------ #### [ Rust ] **See the complete code sample**: [beacon\$1config.rs](https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/main/releases/rust/db_esdk/examples/searchableencryption/complexexample/beacon_config.rs) **Compound beacon configuration** The following example defines encrypted and signed parts lists locally within the compound beacon configuration. ``` let compound_beacon_list = vec![ CompoundBeacon::builder() .name("compound_beacon_name") .split(".") .encrypted(encrypted_parts_list) .signed(signed_parts_list) .constructors(constructor_list) .build()? ``` **Beacon version definition** The following example defines encrypted and signed parts lists globally in the beacon version. For more information on defining the beacon version, see [Using beacons](using-beacons.md). ``` let beacon_versions = BeaconVersion::builder() .standard_beacons(standard_beacon_list) .compound_beacons(compound_beacon_list) .encrypted_parts(encrypted_parts_list) .signed_parts(signed_parts_list) .version(1) // MUST be 1 .key_store(key_store.clone()) .key_source(BeaconKeySource::Single( SingleKeyStore::builder() .key_id(branch_key_id) .cache_ttl(6000) .build()?, )) .build()?; let beacon_versions = vec![beacon_versions]; ``` ------ You can define your [encrypted parts](#encrypted-parts) and [signed parts](#signed-parts) in locally or globally defined lists. We recommend defining your encrypted and signed parts in a global list in the [beacon version](using-beacons.md#beacon-version) whenever possible. By defining encrypted and signed parts globally, you can define each part once and then reuse the parts in multiple compound beacon configurations. If you only intend to use an encrypted or signed part once, you can define it in a local list in the compound beacon configuration. You can reference both local and global parts in your [constructor list](#constructor-parts). If you define your encrypted and signed parts lists globally, you must provide a list of constructor parts that identify all of the possible ways the compound beacon can assemble the fields in your compound beacon configuration. **Note** To define encrypted and signed parts lists globally, you must use version 3.2 or later of the AWS Database Encryption SDK. Deploy the new version to all readers before defining any new parts globally. You cannot update existing beacon configurations to define encrypted and signed parts lists globally. To configure a compound beacon, provide the following values. **Beacon name** The name you use when querying an encrypted field. A beacon name can be the same name as an encrypted field or virtual field, but it cannot be the same name as an unencrypted field. No two beacons can have the same beacon name. For help determining the best beacon name for your implementation, see [Choosing a beacon name](choosing-beacon-name.md). **Split character** The character used to separate the parts that make up your compound beacon. The split character cannot appear in the plaintext values of any of the fields that the compound beacon is constructed from. **Encrypted parts list** Identifies the `ENCRYPT_AND_SIGN` fields included in the compound beacon. Each part must include a name and prefix. The part name must be the name of the standard beacon constructed from the encrypted field. The prefix can be any string, but it must be unique. An encrypted part cannot have the same prefix as a signed part. We recommend using a short value that distinguishes the part from other parts served by the compound beacon. We recommend defining your encrypted parts globally whenever possible. You might consider defining an encrypted part locally if you only intend on using it in one compound beacon. A locally defined encrypted part cannot have the same prefix or name as a globally defined encrypted part. ``` List encryptedPartList = new ArrayList<>(); EncryptedPart encryptedPartExample = EncryptedPart.builder() .name("standardBeaconName") .prefix("E-") .build(); encryptedPartList.add(encryptedPartExample); ``` ``` var encryptedPartList = new List(); var encryptedPartExample = new EncryptedPart { Name = "compoundBeaconName", Prefix = "E-" }; encryptedPartList.Add(encryptedPartExample); ``` ``` let encrypted_parts_list = vec![ EncryptedPart::builder() .name("standard_beacon_name") .prefix("E-") .build()? ]; ``` **Signed parts list** Identifies the signed fields included in the compound beacon. Signed parts are optional. You can configure a compound beacon that does not reference any signed parts. Each part must include a name, source, and prefix. The source is the `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` field that the part identifies. The source must be a field name or an index referring to the value of a nested field. If your part name identifies the source, you can omit the source and the AWS Database Encryption SDK will automatically use the name as its source. We recommend specifying the source as the part name whenever possible. The prefix can be any string, but it must be unique. A signed part cannot have the same prefix as an encrypted part. We recommend using a short value that distinguishes the part from other parts served by the compound beacon. We recommend defining your signed parts globally whenever possible. You might consider defining a signed part locally if you only intend on using it in one compound beacon. A locally defined signed part cannot have the same prefix or name as a globally defined signed part. ``` List signedPartList = new ArrayList<>(); SignedPart signedPartExample = SignedPart.builder() .name("signedFieldName") .prefix("S-") .build(); signedPartList.add(signedPartExample); ``` ``` var signedPartsList = new List { new SignedPart { Name = "signedFieldName1", Prefix = "S-" }, new SignedPart { Name = "signedFieldName2", Prefix = "SF-" } }; ``` ``` let signed_parts_list = vec![ SignedPart::builder() .name("signed_field_name_1") .prefix("S-") .build()?, SignedPart::builder() .name("signed_field_name_2") .prefix("SF-") .build()?, ]; ``` **Constructor list** Identifies the *constructors* that define the different ways that the encrypted and signed parts can be assembled by the compound beacon. You can reference both local and global parts in your constructor list. If you construct your compound beacon from globally defined encrypted and signed parts, you must provide a constructor list. If you do not use any globally defined encrypted or signed parts to construct your compound beacon, the constructor list is optional. If you do not specify a constructor list, the AWS Database Encryption SDK assembles the compound beacon with the following default constructor. + All signed parts in the order they were added to the signed parts list + All encrypted parts in the order they were added to the encrypted parts list + All parts are required **Constructors** Each constructor is an ordered list of *constructor parts* that defines one way that the compound beacon can be assembled. The constructor parts are joined together in the order they are added to the list, with each part separated by the specified split character. Each constructor part names an encrypted part or a signed part, and defines whether that part is required or optional within the constructor. For example, if you want to query a compound beacon on `Field1`, `Field1.Field2`, and `Field1.Field2.Field3`, mark `Field2` and `Field3` as optional and create one constructor. Each constructor must have at least one required part. We recommend making the first part in each constructor required so that you can use the `BEGINS_WITH` operator in your queries. A constructor succeeds if all its required parts are present in the record. When you write a new record, the compound beacon uses the constructor list to determine if the beacon can be assembled from the values provided. It attempts to assemble the beacon in the order that the constructors were added to the constructor list, and it uses the first constructor that succeeds. If no constructors succeed, the beacon is not written to the record. All readers and writers should specify the same order of constructors to ensure that their query results are correct. Use the following procedures to specify your own constructor list. 1. Create a constructor part for each encrypted part and signed part to define whether or not that part is required. The constructor part name must be the name of the standard beacon or signed field it represents. ------ #### [ Java ] ``` ConstructorPart field1ConstructorPart = ConstructorPart.builder() .name("Field1") .required(true) .build(); ``` ------ #### [ C\$1 / .NET ] ``` var field1ConstructorPart = new ConstructorPart { Name = "Field1", Required = true }; ``` ------ #### [ Rust ] ``` let field_1_constructor_part = ConstructorPart::builder() .name("field_1") .required(true) .build()?; ``` ------ 1. Create a constructor for each possible way that the compound beacon can be assembled using the constructor parts you created in **Step 1**. For example, if you want to query on `Field1.Field2.Field3` and `Field4.Field2.Field3`, then you must create two constructors. `Field1` and `Field4` can both be required because they are defined in two separate constructors. ------ #### [ Java ] ``` // Create a list for Field1.Field2.Field3 queries List field123ConstructorPartList = new ArrayList<>(); field123ConstructorPartList.add(field1ConstructorPart); field123ConstructorPartList.add(field2ConstructorPart); field123ConstructorPartList.add(field3ConstructorPart); Constructor field123Constructor = Constructor.builder() .parts(field123ConstructorPartList) .build(); // Create a list for Field4.Field2.Field1 queries List field421ConstructorPartList = new ArrayList<>(); field421ConstructorPartList.add(field4ConstructorPart); field421ConstructorPartList.add(field2ConstructorPart); field421ConstructorPartList.add(field1ConstructorPart); Constructor field421Constructor = Constructor.builder() .parts(field421ConstructorPartList) .build(); ``` ------ #### [ C\$1 / .NET ] ``` // Create a list for Field1.Field2.Field3 queries var field123ConstructorPartList = new Constructor { Parts = new List { field1ConstructorPart, field2ConstructorPart, field3ConstructorPart } }; // Create a list for Field4.Field2.Field1 queries var field421ConstructorPartList = new Constructor { Parts = new List { field4ConstructorPart, field2ConstructorPart, field1ConstructorPart } }; ``` ------ #### [ Rust ] ``` // Create a list for field1.field2.field3 queries let field1_field2_field3_constructor = Constructor::builder() .parts(vec![ field1_constructor_part, field2_constroctor_part.clone(), field3_constructor_part, ]) .build()?; // Create a list for field4.field2.field1 queries let field4_field2_field1_constructor = Constructor::builder() .parts(vec![ field4_constructor_part, field2_constroctor_part.clone(), field1_constructor_part, ]) .build()?; ``` ------ 1. Create a constructor list that includes all of the constructors that you created in **Step 2**. ------ #### [ Java ] ``` List constructorList = new ArrayList<>(); constructorList.add(field123Constructor); constructorList.add(field421Constructor); ``` ------ #### [ C\$1 / .NET ] ``` var constructorList = new List { field123Constructor, field421Constructor }; ``` ------ #### [ Rust ] ``` let constructor_list = vec![ field1_field2_field3_constructor, field4_field2_field1_constructor, ]; ``` ------ 1. Specify the `constructorList` when you create your compound beacon. # Example configurations **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | The following examples demonstrate how to configure standard and compound beacons. The following configurations do not provide beacon lengths. For help determining the appropriate beacon length for your configuration, see [Choose a beacon length](choosing-beacon-length.md). To see complete code examples that demonstrate how to configure and use beacons, see the [Java](https://github.com/aws/aws-database-encryption-sdk-dynamodb//tree/main/Examples/runtimes/java/DynamoDbEncryption/src/main/java/software/amazon/cryptography/examples/searchableencryption), [.NET](https://github.com/aws/aws-database-encryption-sdk-dynamodb/tree/main/Examples/runtimes/net/src/searchableencryption/), and [Rust](https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/main/releases/rust/db_esdk/examples/searchableencryption/) searchable encryption examples in the aws-database-encryption-sdk-dynamodb repository on GitHub. **Topics** + [Standard beacons](#standard-config-examples) + [Compound beacons](#compound-config-examples) ## Standard beacons If you want to query the `inspector_id_last4` field for exact matches, create a standard beacon using the following configuration. ------ #### [ Java ] ``` List standardBeaconList = new ArrayList<>(); StandardBeacon exampleStandardBeacon = StandardBeacon.builder() .name("inspector_id_last4") .length(beaconLengthInBits) .build(); standardBeaconList.add(exampleStandardBeacon); ``` ------ #### [ C\$1 / .NET ] ``` var standardBeaconList = new List>); StandardBeacon exampleStandardBeacon = new StandardBeacon { Name = "inspector_id_last4", Length = 10 }; standardBeaconList.Add(exampleStandardBeacon); ``` ------ #### [ Rust ] ``` let last4_beacon = StandardBeacon::builder() .name("inspector_id_last4") .length(10) .build()?; let unit_beacon = StandardBeacon::builder().name("unit").length(30).build()?; let standard_beacon_list = vec![last4_beacon, unit_beacon]; ``` ------ ## Compound beacons If you want to query the `UnitInspection` database on `inspector_id_last4` and `inspector_id_last4.unit`, create a compound beacon with the following configuration. This compound beacon only requires [encrypted parts](configure-beacons.md#encrypted-parts). ------ #### [ Java ] ``` // 1. Create standard beacons for the inspector_id_last4 and unit fields. List standardBeaconList = new ArrayList<>(); StandardBeacon inspectorBeacon = StandardBeacon.builder() .name("inspector_id_last4") .length(beaconLengthInBits) .build(); standardBeaconList.add(inspectorBeacon); StandardBeacon unitBeacon = StandardBeacon.builder() .name("unit") .length(beaconLengthInBits) .build(); standardBeaconList.add(unitBeacon); // 2. Define the encrypted parts. List encryptedPartList = new ArrayList<>(); // Each encrypted part needs a name and prefix // The name must be the name of the standard beacon // The prefix must be unique // For this example we use the prefix "I-" for "inspector_id_last4" // and "U-" for "unit" EncryptedPart encryptedPartInspector = EncryptedPart.builder() .name("inspector_id_last4") .prefix("I-") .build(); encryptedPartList.add(encryptedPartInspector); EncryptedPart encryptedPartUnit = EncryptedPart.builder() .name("unit") .prefix("U-") .build(); encryptedPartList.add(encryptedPartUnit); // 3. Create the compound beacon. // This compound beacon only requires a name, split character, // and list of encrypted parts CompoundBeacon inspectorUnitBeacon = CompoundBeacon.builder() .name("inspectorUnitBeacon") .split(".") .sensitive(encryptedPartList) .build(); ``` ------ #### [ C\$1 / .NET ] ``` // 1. Create standard beacons for the inspector_id_last4 and unit fields. StandardBeacon inspectorBeacon = new StandardBeacon { Name = "inspector_id_last4", Length = 10 }; standardBeaconList.Add(inspectorBeacon); StandardBeacon unitBeacon = new StandardBeacon { Name = "unit", Length = 30 }; standardBeaconList.Add(unitBeacon); // 2. Define the encrypted parts. var last4EncryptedPart = new EncryptedPart // Each encrypted part needs a name and prefix // The name must be the name of the standard beacon // The prefix must be unique // For this example we use the prefix "I-" for "inspector_id_last4" // and "U-" for "unit" var last4EncryptedPart = new EncryptedPart { Name = "inspector_id_last4", Prefix = "I-" }; encryptedPartList.Add(last4EncryptedPart); var unitEncryptedPart = new EncryptedPart { Name = "unit", Prefix = "U-" }; encryptedPartList.Add(unitEncryptedPart); // 3. Create the compound beacon. // This compound beacon only requires a name, split character, // and list of encrypted parts var compoundBeaconList = new List>); var inspectorCompoundBeacon = new CompoundBeacon { Name = "inspector_id_last4", Split = ".", Encrypted = encryptedPartList }; compoundBeaconList.Add(inspectorCompoundBeacon); ``` ------ #### [ Rust ] ``` // 1. Create standard beacons for the inspector_id_last4 and unit fields. let last4_beacon = StandardBeacon::builder() .name("inspector_id_last4") .length(10) .build()?; let unit_beacon = StandardBeacon::builder().name("unit").length(30).build()?; let standard_beacon_list = vec![last4_beacon, unit_beacon]; // 2. Define the encrypted parts. // The name must be the name of the standard beacon // The prefix must be unique // For this example we use the prefix "I-" for "inspector_id_last4" // and "U-" for "unit" let encrypted_parts_list = vec![ EncryptedPart::builder() .name("inspector_id_last4") .prefix("I-") .build()?, EncryptedPart::builder().name("unit").prefix("U-").build()?, ]; // 3. Create the compound beacon // This compound beacon only requires a name, split character, // and list of encrypted parts let compound_beacon_list = vec![CompoundBeacon::builder() .name("last4UnitCompound") .split(".") .encrypted(encrypted_parts_list) .build()?]; ``` ------ # Using beacons **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | Beacons enable you to search encrypted records without decrypting the entire database being queried. Beacons are designed to be implemented in new, unpopulated databases. Any beacon configured in an existing database will only map new records written to the database. Beacons are calculated from the plaintext value of a field, once the field is encrypted there is no way for the beacon to map existing data. After you have written new records with a beacon, you cannot update the beacon's configuration. However, you can add add new beacons for new fields that you add to your record. After you configure your beacons, you must complete the following steps before you begin populating your database and performing queries on your beacons. 1. **Create an AWS KMS Hierarchical keyring** To use searchable encryption, you must use the [AWS KMS Hierarchical keyring](use-hierarchical-keyring.md) to generate, encrypt, and decrypt the [data keys](concepts.md#data-key) used to protect your records. After you configure your beacons, assemble the [Hierarchical keyring prerequisites](use-hierarchical-keyring.md#hierarchical-keyring-prereqs) and [create your Hierarchical keyring](use-hierarchical-keyring.md#initialize-hierarchical-keyring). For more details on why the Hierarchical keyring is required, see [Using the Hierarchical keyring for searchable encryption](use-hierarchical-keyring.md#searchable-encryption-hierarchical-keyrings). 1. **Define the beacon version ** Specify your `keyStore`, `keySource`, a list of all standard beacons you configured, a list of all compound beacons you configured, a list of encrypted parts, a list of signed parts, and a beacon version. You must specify `1` for the beacon version. For guidance on defining your `keySource`, see [Defining your beacon key source](use-hierarchical-keyring.md#beacon-key-source). The following Java example defines the beacon version for a single tenant database. For help defining the beacon version for a multitenant database, see [Searchable encryption for multitenant databases](searchable-encryption-multitenant.md). ------ #### [ Java ] ``` List beaconVersions = new ArrayList<>(); beaconVersions.add( BeaconVersion.builder() .standardBeacons(standardBeaconList) .compoundBeacons(compoundBeaconList) .encryptedParts(encryptedPartsList) .signedParts(signedPartsList) .version(1) // MUST be 1 .keyStore(keyStore) .keySource(BeaconKeySource.builder() .single(SingleKeyStore.builder() .keyId(branchKeyId) .cacheTTL(6000) .build()) .build()) .build() ); ``` ------ #### [ C\$1 / .NET ] ``` var beaconVersions = new List { new BeaconVersion { StandardBeacons = standardBeaconList, CompoundBeacons = compoundBeaconList, EncryptedParts = encryptedPartsList, SignedParts = signedPartsList, Version = 1, // MUST be 1 KeyStore = branchKeyStoreName, KeySource = new BeaconKeySource { Single = new SingleKeyStore { KeyId = branch-key-id, CacheTTL = 6000 } } } }; ``` ------ #### [ Rust ] ``` let beacon_version = BeaconVersion::builder() .standard_beacons(standard_beacon_list) .compound_beacons(compound_beacon_list) .version(1) // MUST be 1 .key_store(key_store.clone()) .key_source(BeaconKeySource::Single( SingleKeyStore::builder() // `keyId` references a beacon key. // For every branch key we create in the keystore, // we also create a beacon key. // This beacon key is not the same as the branch key, // but is created with the same ID as the branch key. .key_id(branch_key_id) .cache_ttl(6000) .build()?, )) .build()?; let beacon_versions = vec![beacon_version]; ``` ------ 1. **Configure secondary indexes** After you [configure your beacons](configure-beacons.md), you must configure a secondary index that reflects each beacon before you can search on the encrypted fields. For more information, see [Configuring secondary indexes with beacons](ddb-searchable-encryption.md#ddb-beacon-indexes). 1. **Define your [cryptographic actions](concepts.md#crypt-actions)** All fields used to construct a standard beacon must be marked `ENCRYPT_AND_SIGN`. All other fields used to construct beacons must be marked `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT`. 1. **Configure an AWS Database Encryption SDK client** To configure an AWS Database Encryption SDK client that protects the table items in your DynamoDB table, see [Java client-side encryption library for DynamoDB](ddb-java.md). ## Querying beacons The type of beacon you configure determines the type of queries you are able to perform. Standard beacons use filter expressions to perform equality searches. Compound beacons combine literal plaintext strings and standard beacons to perform complex queries. When you query encrypted data, you search on the beacon name. You cannot compare the values of two standard beacons, even if they contain the same underlying plaintext. The two standard beacons will produce two different HMAC tags for the same plaintext values. As a result, standard beacons cannot perform the following queries. + `beacon1 = beacon2` + `beacon1 IN (beacon2)` + `value IN (beacon1, beacon2, ...)` + `CONTAINS(beacon1, beacon2)` Compound beacons can perform the following queries. + `BEGINS_WITH(a)`, where `a` reflects the entire value of the field that the assembled compound beacon begins with. You cannot use the `BEGINS_WITH` operator to identify a value that begins with a particular substring. However, you can use `BEGINS_WITH(S_)`, where `S_` reflects the prefix for a part that the assembled compound beacon begins with. + `CONTAINS(a)`, where `a` reflects the entire value of a field that the assembled compound beacon contains. You cannot use the `CONTAINS` operator to identify a record that contains a particular substring or a value within a set. For example, you cannot perform a query `CONTAINS(path, "a"` where `a` reflects the value in a set. + You can compare [signed parts](configure-beacons.md#signed-parts) of compound beacons. When you compare signed parts, you can optionally append the prefix of an [encrypted part](configure-beacons.md#encrypted-parts) to one or more signed parts, but you cannot include the value of an encrypted field in any query. For example, you can compare signed parts and query on `signedField1 = signedField2` or `value IN (signedField1, signedField2, ...)`. You can also compare signed parts and the prefix of an encrypted part by query on `signedField1.A_ = signedField2.B_`. + `field BETWEEN a AND b`, where `a` and `b` are signed parts. You can optionally append the prefix of an encrypted part to one or more signed parts, but you cannot include the value of an encrypted field in any query. You must include the prefix for each part you include in a query on a compound beacon. For example, if you constructed a compound beacon, `compoundBeacon`, from two fields, `encryptedField` and `signedField`, you must include the prefixes configured for those two parts when you query the beacon. ``` compoundBeacon = E_encryptedFieldValue.S_signedFieldValue ``` # Searchable encryption for multitenant databases **** | | | --- | | Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the [DynamoDB Encryption Client](legacy-dynamodb-encryption-client.md). | To implement searchable encryption in your database, you must use an [AWS KMS Hierarchical keyring](use-hierarchical-keyring.md). The AWS KMS Hierarchical keyring generates, encrypts, and decrypts the data keys used to protect your records. It also creates the beacon key used to generate beacons. When using the AWS KMS Hierarchical keyring with multitenant databases, there is a distinct branch key and beacon key for each tenant. To query encrypted data in a multitenant database, you must identify the beacon key materials used to generate the beacon you are querying. For more information, see [Using the Hierarchical keyring for searchable encryption](use-hierarchical-keyring.md#searchable-encryption-hierarchical-keyrings). When you define the [beacon version](using-beacons.md#beacon-version) for a multitenant database, specify a list of all standard beacons you configured, a list of all compound beacons you configured, a beacon version, and a `keySource`. You must [define your beacon key source](use-hierarchical-keyring.md#beacon-key-source) as a `MultiKeyStore`, and include a `keyFieldName`, a cache time to live for the local beacon key cache, and maximum cache size for the local beacon key cache. If you configured any [signed beacons](configure.md#signed-beacons), they must be included in your `compoundBeaconList`. Signed beacons are a type of compound beacon that index and perform complex queries on `SIGN_ONLY` and `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT`fields. ------ #### [ Java ] ``` List beaconVersions = new ArrayList<>(); beaconVersions.add( BeaconVersion.builder() .standardBeacons(standardBeaconList) .compoundBeacons(compoundBeaconList) .version(1) // MUST be 1 .keyStore(branchKeyStoreName) .keySource(BeaconKeySource.builder() .multi(MultiKeyStore.builder() .keyFieldName(keyField) .cacheTTL(6000) .maxCacheSize(10) .build()) .build()) .build() ); ``` ------ #### [ C\$1 / .NET ] ``` var beaconVersions = new List { new BeaconVersion { StandardBeacons = standardBeaconList, CompoundBeacons = compoundBeaconList, EncryptedParts = encryptedPartsList, SignedParts = signedPartsList, Version = 1, // MUST be 1 KeyStore = branchKeyStoreName, KeySource = new BeaconKeySource { Multi = new MultiKeyStore { KeyId = branch-key-id, CacheTTL = 6000, MaxCacheSize = 10 } } } }; ``` ------ #### [ Rust ] ``` let beacon_version = BeaconVersion::builder() .standard_beacons(standard_beacon_list) .compound_beacons(compound_beacon_list) .version(1) // MUST be 1 .key_store(key_store.clone()) .key_source(BeaconKeySource::Multi( MultiKeyStore::builder() // `keyId` references a beacon key. // For every branch key we create in the keystore, // we also create a beacon key. // This beacon key is not the same as the branch key, // but is created with the same ID as the branch key. .key_id(branch_key_id) .cache_ttl(6000) .max_cache_size(10) .build()?, )) .build()?; let beacon_versions = vec![beacon_version]; ``` ------ **keyFieldName** The [`keyFieldName`](use-hierarchical-keyring.md#keyFieldName) defines the name of the field that stores the `branch-key-id` associated with the beacon key used to generated beacons for a given tenant. When you write new records to your database, the `branch-key-id` that identifies the beacon key used to generate any beacons for that record is stored in this field. By default, the `keyField` is a conceptual field that is not explicitly stored in your database. The AWS Database Encryption SDK identifies the `branch-key-id` from the encrypted [data key](concepts.md#data-key) in the [material description](concepts.md#material-description) and stores the value in the conceptual `keyField` for you to reference in your compound beacons and [signed beacons](configure.md#signed-beacons). Since the material description is signed, the conceptual `keyField` is considered a signed part. You can also include the `keyField` in your cryptographic actions as a `SIGN_ONLY` or `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` field to explicitly store the field in your database. If you do this, you must manually include the `branch-key-id` in the `keyField` every time you write a record to your database. ## Querying beacons in a multitenant database To query a beacon, you must include the `keyField` in your query to identify the appropriate beacon key materials required to recalculate the beacon. You must specify the `branch-key-id` associated with the beacon key used to generate the beacons for a record. You cannot specify the [friendly name](use-hierarchical-keyring.md#branch-key-id-supplier) that identifies a tenant's `branch-key-id` in the branch key ID supplier. You can include the `keyField` in your queries in a following ways. **Compound beacons** Whether you explicitly store the `keyField` in your records or not, you can include the `keyField` directly in your compound beacons as a signed part. The `keyField` signed part must be required. For example, if you want to construct a compound beacon, `compoundBeacon`, from two fields, `encryptedField` and `signedField`, you must also include the `keyField` as a signed part. This enables you to perform the following query on `compoundBeacon`. ``` compoundBeacon = E_encryptedFieldValue.S_signedFieldValue.K_branch-key-id ``` **Signed beacons** The AWS Database Encryption SDK uses standard and compound beacons to provide searchable encryption solutions. These beacons must include at least one encrypted field. However, the AWS Database Encryption SDK also supports [signed beacons](configure.md#signed-beacons) that can be configured entirely from plaintext `SIGN_ONLY` and `SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT` fields. Signed beacons can be constructed from a single part. Whether you explicitly store the `keyField` in your records or not, you can construct a signed beacon from the `keyField` and use it to create compound queries that combine a query on the `keyField` signed beacon with a query on one of your other beacons. For example, you could perform the following query. ``` keyField = K_branch-key-id AND compoundBeacon = E_encryptedFieldValue.S_signedFieldValue ``` For help configuring signed beacons, see [Creating signed beacons](configure.md#signed-beacons) **Query directly on the `keyField`** If you specified the `keyField` in your cryptographic actions and explicitly store the field in your record, you can create a compound query that combines a query on your beacon with a query on the `keyField`. You might choose to query directly on the `keyField` if you want to query a standard beacon. For example, you could perform the following query. ``` keyField = branch-key-id AND standardBeacon = S_standardBeaconValue ```