Create a custom connector to a data source

This topic describes how to connect a custom data source to CloudWatch. You can connect a custom data source to CloudWatch in two ways:

Use a template

Using a template creates a sample Lambda function, and can help you get your custom connector built faster. These sample functions provide sample code for many common scenarios involved with building a custom connector. You can examine the Lambda code after you create a connector with a template, then modify it to use to connect to your data source.

Additionally, if you use the template, CloudWatch takes care of creating the Lambda permissions policy and attaching resource tags to the Lambda function.

Create a custom data source from scratch

Follow the steps in this section to create a Lambda function that connects CloudWatch to a data source.

Step 1: Create the function

A custom data source connector must support GetMetricData events from CloudWatch. Optionally, you can also implement a DescribeGetMetricData event to provide documentation to users in the CloudWatch console for how to use the connector. The DescribeGetMetricData response can also be used to set defaults that are used in the CloudWatch custom query builder.

CloudWatch provides code snippets as samples to help you get started. For more information, see the samples repository at https://github.com/aws-samples/cloudwatch-data-source-samples.

Constraints

GetMetricData event

Request payload

The following is an example of a GetMetricData request payload sent as input to the Lambda function.

{
  "EventType": "GetMetricData",
  "GetMetricDataRequest": {
    "StartTime": 1697060700,
    "EndTime": 1697061600,
    "Period": 300,
    "Arguments": ["serviceregistry_external_http_requests{host_cluster!=\"prod\"}"] 
  }
}

Response payload

The following is an example of a GetMetricData response payload returned by the Lambda function.

{
   "MetricDataResults": [
      {
         "StatusCode": "Complete",
         "Label": "CPUUtilization",
         "Timestamps": [ 1697060700, 1697061000, 1697061300 ],
         "Values": [ 15000, 14000, 16000 ]
      }
   ]
}

The response payload will contain either a MetricDataResults field or an Error field, but not both.

A MetricDataResults field is a list of time-series fields of type MetricDataResult. Each of those time-series fields can include the following fields.

For more information about Error objects, see the following sections.

Error response formats

You can optionally use the error response to provide more information about errors. We recommend that you return an error with Code Validation when a validation error occurs, such as when a parameter is missing or is the wrong type.

The following is an example of the response when the Lambda function wants to raise a GetMetricData validation exception.

{
   "Error": {
      "Code": "Validation",
      "Value": "Invalid Prometheus cluster"
   }
}

The following is an example of the response when the Lambda function indicates that it's unable to return data because of an access issue. The response is translated into a single time series with a status code of Forbidden.

{
   "Error": {
      "Code": "Forbidden",
      "Value": "Unable to access ..."
   }
}

The following is an example of when the Lambda function raises an overall InternalError exception, which is translated into a single time series with a status code of InternalError and a message. Whenever an error code has a value other than Validation or Forbidden, CloudWatch assumes that it's a generic internal error.

{
   "Error": {
      "Code": "PrometheusClusterUnreachable",
      "Value": "Unable to communicate with the cluster"
   }
}

DescribeGetMetricData event

Request payload

The following is an example of a DescribeGetMetricData request payload.

{
  "EventType": "DescribeGetMetricData"
}

Response payload

The following is an example of a DescribeGetMetricData response payload.

{
    "Description": "Data source connector",
    "ArgumentDefaults": [{
        Value: "default value"
     }]
}

Important considerations for CloudWatch alarms

If you are going to use the data source to set CloudWatch alarms, you should set it up to report data with timestamps every minute to CloudWatch. For more information and other considerations for creating alarms on metrics from connected data sources, see Create an alarm based on a connected data source.

(Optional) Use AWS Secrets Manager to store credentials

If your Lambda function needs to use credentials to access the data source, we recommend using AWS Secrets Manager to store these credentials instead of hardcoding them into your Lambda function. For more information about using AWS Secrets Manager with Lambda, see Use AWS Secrets Manager secrets in AWS Lambda functions.

(Optional) Connect to a data source in a VPC

If your data source is in a VPC managed by Amazon Virtual Private Cloud, you must configure your Lambda function to access it. For more information, see Connecting outbound networking to resources in a VPC.

You might also need to configure VPC service endpoints to access services such as AWS Secrets Manager. For more information, see Access an AWS service using an interface VPC endpoint.

Step 2: Create a Lambda permissions policy

You must use create a policy statement that grants CloudWatch permission to use the Lambda function that you created. You can use the AWS CLI or the Lambda console to create the policy statement.

Step 3: Attach a resource tag to the Lambda function

The CloudWatch console determines which of your Lambda functions are data source connectors by using a tag. When you create a data source using one of the wizards, the tag is automatically applied by the AWS CloudFormation stack that configures it. When you create a data source yourself, you can use the following tag for your Lambda function. This makes your connector appear in the Data source dropdown in the CloudWatch console when you query metrics.