Tutorial: Externalizing sensitive information using config providers

Open the Secrets Manager console.

Create a new secret to store your credentials or secrets. For instructions, see Create an AWS Secrets Manager secret in the AWS Secrets Manager User Guide.

Copy your secret's ARN.

Add the Secrets Manager permissions from the following example policy to your Service execution role. Replace <arn:aws:secretsmanager:us-east-1:123456789000:secret:MySecret-1234> with the ARN of your secret.

Add worker configuration and connector instructions.

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "secretsmanager:GetResourcePolicy",
                    "secretsmanager:GetSecretValue",
                    "secretsmanager:DescribeSecret",
                    "secretsmanager:ListSecretVersionIds"
                ],
                "Resource": [
                "<arn:aws:secretsmanager:us-east-1:123456789000:secret:MySecret-1234>"
                ]
            }
        ]
    }

For using the Secrets Manager configuration provider, copy the following lines of code to the worker configuration textbox in Step 3:

# define name of config provider:

config.providers = secretsmanager

# provide implementation classes for secrets manager:

config.providers.secretsmanager.class = com.amazonaws.kafka.config.providers.SecretsManagerConfigProvider

# configure a config provider (if it needs additional initialization), for example you can provide a region where the secrets or parameters are located:

config.providers.secretsmanager.param.region = us-east-1

For the secrets manager configuration provider, copy the following lines of code in the connector configuration in Step 4.

#Example implementation for secrets manager variable
database.hostname=${secretsmanager:MSKAuroraDBCredentials:username}

database.password=${secretsmanager:MSKAuroraDBCredentials:password}

Open the Systems Manager console.

In the navigation pane, choose Parameter Store.

Create a new parameter to store in the Systems Manager. For instructions, see Create a Systems Manager parameter (console) in the AWS Systems Manager User Guide.

Copy your parameter's ARN.

Add the Systems Manager permissions from the following example policy to your Service execution role. Replace <arn:aws:ssm:us-east-1:123456789000:parameter/MyParameterName> with the ARN of your parameter.

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "ssm:GetParameterHistory",
                    "ssm:GetParametersByPath",
                    "ssm:GetParameters",
                    "ssm:GetParameter"
                ],
                "Resource": "arn:aws:ssm:us-east-1:123456789000:parameter/MyParameterName"
            }
        ]
    }

For using the parameter store configuration provider, copy the following lines of code to the worker configuration textbox in Step 3:

# define name of config provider:

config.providers = ssm

# provide implementation classes for parameter store:

config.providers.ssm.class = com.amazonaws.kafka.config.providers.SsmParamStoreConfigProvider

# configure a config provider (if it needs additional initialization), for example you can provide a region where the secrets or parameters are located:

config.providers.ssm.param.region = us-east-1

For the parameter store configuration provider copy the following lines of code in the connector configuration in Step 5.

#Example implementation for parameter store variable
schema.history.internal.kafka.bootstrap.servers=${ssm::MSKBootstrapServerAddress}

You may also bundle the above two steps with more configuration providers.

Open the Amazon S3 console.

Upload your object to a bucket in S3. For instructions, see Uploading objects.

Copy your object's ARN.

Add the Amazon S3 Object Read permissions from the following example policy to your Service execution role. Replace <arn:aws:s3:::MY_S3_BUCKET/path/to/custom-plugin.zip> with the ARN of your object.

{
    "Version": "2012-10-17",
    "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": "s3:GetObject",
                "Resource": "<arn:aws:s3:::MY_S3_BUCKET/path/to/custom-plugin.zip>"
            }
        ]
    }

For using the Amazon S3 configuration provider, copy the following lines of code to the worker configuration text-box in Step 3:

# define name of config provider:

config.providers = s3import
# provide implementation classes for S3:

config.providers.s3import.class = com.amazonaws.kafka.config.providers.S3ImportConfigProvider

For the Amazon S3 configuration provider, copy the following lines of code to the connector configuration in Step 4.

#Example implementation for S3 object

database.ssl.truststore.location = ${s3import:us-west-2:my_cert_bucket/path/to/trustore_unique_filename.jks}

You may also bundle the above two steps with more configuration providers.