Create tag-based Amazon CloudWatch dashboards automatically - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsBest practicesEpicsTroubleshootingRelated resourcesAdditional information

Create tag-based Amazon CloudWatch dashboards automatically

Created by Janak Vadaria (AWS), RAJNEESH TYAGI (AWS), and Vinodkumar Mandalapu (AWS)

Code repository: Goldensignals

Environment: Production

Technologies: Operations; CloudNative; Management & governance

AWS services: AWS CDK; Amazon CloudWatch; AWS CodeBuild; AWS CodePipeline

Summary

Creating different Amazon CloudWatch dashboards manually can be time-consuming, particularly when you have to create and update multiple resources to automatically scale your environment. A solution that creates and updates your CloudWatch dashboards automatically can save you time. This pattern helps you deploy a fully automated AWS Cloud Development Kit (AWS CDK) pipeline that creates and updates CloudWatch dashboards for your AWS resources based on tag change events, to display Golden Signals metrics.

In site reliability engineering (SRE), Golden Signals refers to a comprehensive set of metrics that offer a broad view of a service from a user or consumer perspective. These metrics consist of latency, traffic, errors, and saturation. For more information, see What is Site Reliability Engineering (SRE)? on the AWS website.

The solution provided by this pattern is event-driven. After it's deployed, it continuously monitors the tag change events and automatically updates the CloudWatch dashboards and alarms.

Prerequisites and limitations

Prerequisites

Limitations

This solution currently creates automated dashboards for the following AWS services only:

Architecture

Target technology stack

Target architecture

Target architecture for creating tag-based CloudWatch dashboards

  1. An AWS tag change event for the configured application tags or code changes initiates a pipeline in AWS CodePipeline to build and deploy updated CloudWatch dashboards.

  2. AWS CodeBuild runs a Python script to find the resources that have configured tags and stores the resource IDs in a local file in a CodeBuild environment.

  3. CodeBuild runs cdk synth to generate AWS CloudFormation templates that deploy CloudWatch dashboards and alarms.

  4. CodePipeline deploys the AWS CloudFormation templates to the specified AWS account and Region.

  5. When the AWS CloudFormation stack has been deployed successfully, you can view the CloudWatch dashboards and alarms.

Automation and scale

This solution has been automated by using the AWS CDK. You can find the code in the GitHub Golden Signals Dashboards on Amazon CloudWatch repository. For additional scaling and to create custom dashboards, you can configure multiple tag keys and values.

Tools

Amazon services

  • Amazon EventBridge is a serverless event bus service that helps you connect your applications with real-time data from a variety of sources, including AWS Lambda functions, HTTP invocation endpoints using API destinations, or event buses in other AWS accounts.

  • AWS CodePipeline helps you quickly model and configure the different stages of a software release and automate the steps required to release software changes continuously.

  • AWS CodeBuild is a fully managed build service that helps you compile source code, run unit tests, and produce artifacts that are ready to deploy.

  • AWS CodeCommit is a version control service that helps you privately store and manage Git repositories without needing to manage your own source control system.

  • AWS Command Line Interface (AWS CLI) is an open source tool that helps you interact with AWS services through commands in your command-line shell.

  • AWS Identity and Access Management (IAM) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them.

  • Amazon Simple Storage Service (Amazon S3) is a cloud-based object storage service that helps you store, protect, and retrieve any amount of data.

Best practices

As a security best practice, you can use encryption and authentication for the source repositories that connect to your pipelines. For additional best practices, see CodePipeline best practices and use cases in the CodePipeline documentation.

Epics

TaskDescriptionSkills required

Configure and deploy the sample application.

  1. Clone the GitHub sample code repository by using the command:

    git clone https://github.com/aws-samples/golden-signals-dashboards-sample-app

  2. Navigate to the cloned repository on your computer and open the src/project-settings.ts file with the editor of your choice.

  3. Change the projectSettings constant value according to your AWS resource tags and application mappings.

  4. Set the AWS_ACCOUNT, AWS_REGION, and GS_DASHBOARD_INSTANCE environment variables:

    • Set AWS_ACCOUNT to the account ID of your AWS account.

    • Set AWS_REGION to the Region where you want to deploy the sample application.

    • Set GS_DASHBOARD_INSTANCE to dev, test, or prod, depending on your development environment. (We recommend test for the testing procedure described in this pattern.)

  5. Set up the AWS CLI with your AWS credentials. For more information, see Set and view configuration settings using commands in the AWS CLI documentation.

  6. Run the following command to deploy the Golden Signals dashboard sample application:

     sh deploy.sh
AWS DevOps

Automatically create dashboards and alarms.

After you deploy the sample application, you can create any of the resources that this solution supports with expected tag values, which will automatically create the specified dashboards and alarms.

To test this solution, create an AWS Lambda function:

  1. Sign in to the AWS Management Console in the AWS Region where you deployed the sample application.

  2. Open the Lambda console at https://console.aws.amazon.com/lambda/.

  3. Choose Create a function, and then enter a function name.

  4. In the Advanced settings pane, select Enable tags, and then choose Add new tag. Enter the following key and value:

    • Key: AutoDashboard

    • Value: True

  5. Choose Create function.

    The Lambda function immediately starts a code pipeline, which creates the dashboards and alarms for that particular Lambda function automatically.

  6. To view the automated dashboards and alarms, open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/. You can view the custom dashboards and alarms for the function you specified in the projectSettings constant (APP1-lambda by default).

  7. Select the dashboard for the Lambda function to view additional automated dashboards that were created as part of this solution.

  8. Repeat these steps for other services, such as Amazon RDS, Amazon SNS, AWS Auto Scaling, and DynamoDB, to generate the associated dashboards. For an example for Amazon RDS, see the Additional information section.

AWS DevOps
TaskDescriptionSkills required

Remove the golden-signals-dashboard construct.

  1. To remove all the AWS CloudFormation stacks created by the sample application, you have to reconfigure the AWS_ACCOUNT, AWS_REGION, and GS_DASHBOARD_INSTANCE environment variables. The destroy.sh command requires these configurations.

    • AWS_ACCOUNT is the account ID of your AWS account.

    • AWS_REGION is the Region where you deployed your sample application.

    • GS_DASHBOARD_INSTANCE is dev, test, or prod, based on your previous settings.

  2. Set up AWS CLI with your AWS credentials.

  3. Run the following command to remove the sample application and all associated AWS CloudFormation stacks:

    sh destroy.sh
AWS DevOps

Troubleshooting

IssueSolution

Python command not found (referring to findresources.sh, line 8).

Check the version of your Python installation. If you have installed Python version 3, replace python with python3 on line 8 of the resources.sh file, and run the sh deploy.sh command again to deploy the solution.

Related resources

Additional information

The following illustration shows a sample dashboard for Amazon RDS that is created as part of this solution.

Sample dashboard for Amazon RDS