Syntax Properties Return values Examples

AWS::Pipes::Pipe

Specifies a pipe. Amazon EventBridge Pipes connect event sources to targets and reduces the need for specialized knowledge and integration code.

Note

As an aid to help you jumpstart developing CloudFormation templates, the EventBridge console enables you to create templates from the existing pipes in your account. For more information, see Generate an CloudFormation template from EventBridge Pipes in the Amazon EventBridge User Guide.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON


{
  "Type" : "AWS::Pipes::Pipe",
  "Properties" : {
      "Description" : String,
      "DesiredState" : String,
      "Enrichment" : String,
      "EnrichmentParameters" : PipeEnrichmentParameters,
      "KmsKeyIdentifier" : String,
      "LogConfiguration" : PipeLogConfiguration,
      "Name" : String,
      "RoleArn" : String,
      "Source" : String,
      "SourceParameters" : PipeSourceParameters,
      "Tags" : {Key: Value, ...},
      "Target" : String,
      "TargetParameters" : PipeTargetParameters
    }
}

YAML


Type: AWS::Pipes::Pipe
Properties:
  Description: String
  DesiredState: String
  Enrichment: String
  EnrichmentParameters: 
    PipeEnrichmentParameters
  KmsKeyIdentifier: String
  LogConfiguration: 
    PipeLogConfiguration
  Name: String
  RoleArn: String
  Source: String
  SourceParameters: 
    PipeSourceParameters
  Tags: 
    Key: Value
  Target: String
  TargetParameters: 
    PipeTargetParameters

Properties

Description

A description of the pipe.

Required: No

Type: String

Pattern: ^.*$

Minimum: 0

Maximum: 512

Update requires: No interruption

DesiredState

The state the pipe should be in.

Required: No

Type: String

Allowed values: RUNNING | STOPPED

Update requires: No interruption

Enrichment

The ARN of the enrichment resource.

Required: No

Type: String

Pattern: ^$|arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 0

Maximum: 1600

Update requires: No interruption

EnrichmentParameters

The parameters required to set up enrichment on your pipe.

Required: No

Type: PipeEnrichmentParameters

Update requires: No interruption

KmsKeyIdentifier

The identifier of the AWS KMS customer managed key for EventBridge to use, if you choose to use a customer managed key to encrypt pipe data. The identifier can be the key Amazon Resource Name (ARN), KeyId, key alias, or key alias ARN.

To update a pipe that is using the default AWS owned key to use a customer managed key instead, or update a pipe that is using a customer managed key to use a different customer managed key, specify a customer managed key identifier.

To update a pipe that is using a customer managed key to use the default AWS owned key, specify an empty string.

For more information, see Managing keys in the AWS Key Management Service Developer Guide.

Required: No

Type: String

Minimum: 0

Maximum: 2048

Update requires: No interruption

LogConfiguration

The logging configuration settings for the pipe.

Required: No

Type: PipeLogConfiguration

Update requires: No interruption

Name

The name of the pipe.

Required: No

Type: String

Pattern: ^[\.\-_A-Za-z0-9]+$

Minimum: 1

Maximum: 64

Update requires: Replacement

RoleArn

The ARN of the role that allows the pipe to send data to the target.

Required: Yes

Type: String

Pattern: ^arn:(aws[a-zA-Z-]*)?:iam::\d{12}:role/?[a-zA-Z0-9+=,.@\-_/]+$

Minimum: 1

Maximum: 1600

Update requires: No interruption

Source

The ARN of the source resource.

Required: Yes

Type: String

Pattern: ^smk://(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9]):[0-9]{1,5}|arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 1

Maximum: 1600

Update requires: Replacement

SourceParameters

The parameters required to set up a source for your pipe.

Required: No

Type: PipeSourceParameters

Update requires: No interruption

Tags

The list of key-value pairs to associate with the pipe.

Required: No

Type: Object of String

Pattern: .+

Minimum: 0

Maximum: 256

Update requires: No interruption

Target

The ARN of the target resource.

Required: Yes

Type: String

Pattern: ^arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 1

Maximum: 1600

Update requires: No interruption

TargetParameters

The parameters required to set up a target for your pipe.

For more information about pipe target parameters, including how to use dynamic path parameters, see Target parameters in the Amazon EventBridge User Guide.

Required: No

Type: PipeTargetParameters

Update requires: No interruption

Return values

Ref

When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns the name of the pipe that was created by the request.

Fn::GetAtt

The Fn::GetAtt intrinsic function returns a value for a specified attribute of this type. The following are the available attributes and sample return values.

For more information about using the Fn::GetAtt intrinsic function, see Fn::GetAtt.

Arn: The ARN of the pipe.
CreationTime: The time the pipe was created.
CurrentState: The state the pipe is in.
LastModifiedTime: When the pipe was last updated, in ISO-8601 format (YYYY-MM-DDThh:mm:ss.sTZD).
StateReason: The reason the pipe is in its current state.

Create a Pipe with an enrichment

Create a Pipe with an Amazon SQS source, an API Gateway enrichment, and a Step Functions state machine target.

JSON


{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Resources": {
    "TestPipe": {
      "Type": "AWS::Pipes::Pipe",
      "Properties": {
        "Name": "PipeCfnExample",
        "RoleArn": "arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role",
        "Source": "arn:aws:sqs:us-east-1:123456789123:pipeDemoSource",
        "Enrichment": "arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets",
        "Target": "arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine"
      }
    }
  }
}

YAML


AWSTemplateFormatVersion: 2010-09-09
Resources:
  TestPipe:
    Type: AWS::Pipes::Pipe
    Properties:
      Name:  PipeCfnExample
      RoleArn: arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role
      Source: arn:aws:sqs:us-east-1:123456789123:pipeDemoSource
      Enrichment: arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets
      Target: arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine

Create a pipe with an event filter

The following example:

Provisions a DynamoDB table and associated data stream to act as the pipe source, and a Amazon SQS queue for the pipe target.
Provisions an IAM execution role for the pipe that defines the necessary permissions to access both the source and target.
Creates a pipe that connects the DynamoDB stream source to the Amazon SQS queue target.
Within the pipe, defines an event filter with an event pattern that selects events where eventname is INSERT or MODIFY.

Note

Be aware that you will be billed for the AWS resources used if you create a stack from this template.

JSON



{
  "AWSTemplateFormatVersion": "2010-09-09",

 "Description" : "EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB stream as the pipe source and an SQS queue as the pipe target. Also provisions an execution role that contains the necessary permissions to access both the source and target. Once provisioned, the pipe receives events from the DynamoDB data stream, applies a filter, and sends matching events on to an SQS Queue. You will be billed for the Amazon resources used if you create a stack from this template.",

  "Parameters" : {
    "SourceTableName" : {
      "Type" : "String",
      "Default" : "pipe-example-source",
      "Description" : "Specify the name of the table to provision as the pipe source, or accept the default."
    },
  "TargetQueueName" : {
    "Type" : "String",
    "Default" : "pipe-example-target",
    "Description" : "Specify the name of the queue to provision as the pipe target, or accept the default."
  },
    "PipeName" : {
      "Type" : "String",
      "Default" : "pipe-with-filtering-example",
      "Description" : "Specify the name of the table to provision as the pipe source, or accept the default."
    }
},
  "Resources": {
    "PipeSourceDynamoDBTable": {
      "Type": "AWS::DynamoDB::Table",
      "Properties": {
        "AttributeDefinitions": [{
            "AttributeName": "Album",
            "AttributeType": "S"
          },
          {
            "AttributeName": "Artist",
            "AttributeType": "S"
          }

        ],
        "KeySchema": [{
            "AttributeName": "Album",
            "KeyType": "HASH"

          },
          {
            "AttributeName": "Artist",
            "KeyType": "RANGE"
          }
        ],
        "ProvisionedThroughput": {
          "ReadCapacityUnits": 10,
          "WriteCapacityUnits": 10
        },
        "StreamSpecification": {
          "StreamViewType": "NEW_AND_OLD_IMAGES"
        },
        "TableName": { "Ref" : "SourceTableName" }
      }
    },
    "PipeTargetQueue": {
      "Type": "AWS::SQS::Queue",
      "Properties": {
        "QueueName": { "Ref" : "TargetQueueName" }
      }
    },
    "PipeTutorialPipeRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [{
            "Effect": "Allow",
            "Principal": {
              "Service": "pipes.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
              "StringLike": {
                "aws:SourceArn": {
	"Fn::Join": [
		"", 
                [
                  "arn:",
                  { "Ref": "AWS::Partition" },
                  ":pipes:",
                  { "Ref": "AWS::Region" },
                  ":",
                  { "Ref": "AWS::AccountId" },
                  ":pipe/",
                  { "Ref": "PipeName" }
               ]
	]
       },
                "aws:SourceAccount": { "Ref" : "AWS::AccountId" }
              }
            }
          }]
        },
        "Description" : "EventBridge Pipe template example. Execution role that grants the pipe the permissions necessary to send events to the specified pipe.",
        "Path": "/",
        "Policies": [{
            "PolicyName": "SourcePermissions",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [{
                "Effect": "Allow",
                "Action": [
                  "dynamodb:DescribeStream",
                  "dynamodb:GetRecords",
                  "dynamodb:GetShardIterator",
                  "dynamodb:ListStreams"
                ],
                "Resource": [
                  { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] }
                ]
              }]
            }
          },
          {
            "PolicyName": "TargetPermissions",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [{
                "Effect": "Allow",
                "Action": [
                  "sqs:SendMessage"
                ],
                "Resource": [
                  { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] }
                ]
              }]
            }
          }
        ]
      }
  },
    "PipeWithFiltering": {
      "Type": "AWS::Pipes::Pipe",
      "Properties": {
        "Description" : "EventBridge Pipe template example. Pipe that receives events from a DynamoDB stream, applies a filter, and sends matching events on to an SQS Queue.",
        "Name": { "Ref" : "PipeName" },
        "RoleArn": {"Fn::GetAtt" : ["PipeTutorialPipeRole", "Arn"] },
        "Source": { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] },
        "SourceParameters": {
          "DynamoDBStreamParameters" : {
            "StartingPosition" : "LATEST"
         },
        "FilterCriteria" : {
          "Filters" : [ {
            "Pattern" : "{ \"eventName\": [\"INSERT\", \"MODIFY\"] }"
         }]
        }
        },
        "Target": { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] }
      }
    }
  }
}

YAML


AWSTemplateFormatVersion: 2010-09-09
Description: >-
  EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB
  stream as the pipe source and an SQS queue as the pipe target. Also provisions
  an execution role that contains the necessary permissions to access both the
  source and target. Once provisioned, the pipe receives events from the
  DynamoDB data stream, applies a filter, and sends matching events on to an SQS
  Queue. You will be billed for the Amazon resources used if you create a stack
  from this template.
Parameters:
  SourceTableName:
    Type: String
    Default: pipe-example-source
    Description: >-
      Specify the name of the table to provision as the pipe source, or accept
      the default.
  TargetQueueName:
    Type: String
    Default: pipe-example-target
    Description: >-
      Specify the name of the queue to provision as the pipe target, or accept
      the default.
  PipeName:
    Type: String
    Default: pipe-with-filtering-example
    Description: >-
      Specify the name of the table to provision as the pipe source, or accept
      the default.
Resources:
  PipeSourceDynamoDBTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      AttributeDefinitions:
        - AttributeName: Album
          AttributeType: S
        - AttributeName: Artist
          AttributeType: S
      KeySchema:
        - AttributeName: Album
          KeyType: HASH
        - AttributeName: Artist
          KeyType: RANGE
      ProvisionedThroughput:
        ReadCapacityUnits: 10
        WriteCapacityUnits: 10
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      TableName:
        Ref: SourceTableName
  PipeTargetQueue:
    Type: 'AWS::SQS::Queue'
    Properties:
      QueueName:
        Ref: TargetQueueName
  PipeTutorialPipeRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: pipes.amazonaws.com
            Action: 'sts:AssumeRole'
            Condition:
              StringLike:
                'aws:SourceArn':
                  'Fn::Join':
                    - ''
                    - - 'arn:'
                      - Ref: 'AWS::Partition'
                      - ':pipes:'
                      - Ref: 'AWS::Region'
                      - ':'
                      - Ref: 'AWS::AccountId'
                      - ':pipe/'
                      - Ref: PipeName
                'aws:SourceAccount':
                  Ref: 'AWS::AccountId'
      Description: >-
        EventBridge Pipe template example. Execution role that grants the pipe
        the permissions necessary to send events to the specified pipe.
      Path: /
      Policies:
        - PolicyName: SourcePermissions
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'dynamodb:DescribeStream'
                  - 'dynamodb:GetRecords'
                  - 'dynamodb:GetShardIterator'
                  - 'dynamodb:ListStreams'
                Resource:
                  - 'Fn::GetAtt':
                      - PipeSourceDynamoDBTable
                      - StreamArn
        - PolicyName: TargetPermissions
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'sqs:SendMessage'
                Resource:
                  - 'Fn::GetAtt':
                      - PipeTargetQueue
                      - Arn
  PipeWithFiltering:
    Type: 'AWS::Pipes::Pipe'
    Properties:
      Description: >-
        EventBridge Pipe template example. Pipe that receives events from a
        DynamoDB stream, applies a filter, and sends matching events on to an
        SQS Queue.
      Name:
        Ref: PipeName
      RoleArn:
        'Fn::GetAtt':
          - PipeTutorialPipeRole
          - Arn
      Source:
        'Fn::GetAtt':
          - PipeSourceDynamoDBTable
          - StreamArn
      SourceParameters:
        DynamoDBStreamParameters:
          StartingPosition: LATEST
        FilterCriteria:
          Filters:
            - Pattern: '{ "eventName": ["INSERT", "MODIFY"] }'
      Target:
        'Fn::GetAtt':
          - PipeTargetQueue
          - Arn

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Amazon EventBridge Pipes

AwsVpcConfiguration