# Connecting to Amazon S3 source actions with a source enabled for events
The instructions in this section provide the steps for creating the S3 source action that does not require that you create or manage AWS CloudTrail resources.
**Important**
The procedure to create this action without AWS CloudTrail resources is not available in the console. To use the CLI, see the procedures here or see [Migrate polling pipelines with an S3 source enabled for events](update-change-detection.md#update-change-detection-S3-event).
For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge and with a source bucket that is enabled for event notifications. This is the recommend method if you are using the CLI or CloudFormation to migrate your pipeline.
**Note**
This includes using a bucket that is enabled for event notifications, where you do not need to create a separate CloudTrail trail. If you are using the console, then an event rule and CloudTrail trail are set up for you. For those steps, see [Migrate polling pipelines with an S3 source and CloudTrail trail](update-change-detection.md#update-change-detection-S3).
+ **CLI: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CLI)](update-change-detection.md#update-change-detection-cli-S3)
+ **CloudFormation: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CloudFormation template)](update-change-detection.md#update-change-detection-cfn-s3)
# Create pipelines with an S3 source enabled for events (CLI)
Follow these steps to create a pipeline with an S3 source that uses an event in EventBridge for change detection. For the full steps to create a pipeline with the CLI, see [Create a pipeline, stages, and actions](pipelines-create.md).
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then create the following resources:
+ EventBridge event rule
+ IAM role to allow the EventBridge event to start your pipeline
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see [Using resource-based policies for Amazon EventBridge](http://docs.aws.amazon.com/eventbridge/latest/userguide/eb-use-resource-based.html).
1. Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it `trustpolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
1. Use the following command to create the `Role-for-MyRule` role and attach the trust policy.
**Why am I making this change?** Adding this trust policy to the role creates permissions for EventBridge.
```
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
```
1. Create the permissions policy JSON, as shown here for the pipeline named `MyFirstPipeline`. Name the permissions policy `permissionspolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codepipeline:StartPipelineExecution"
],
"Resource": [
"arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline"
]
}
]
}
```
------
1. Use the following command to attach the new `CodePipeline-Permissions-Policy-for-EB` permissions policy to the `Role-for-MyRule` role you created.
```
aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
```
1. Call the **put-rule** command and include the `--name`, `--event-pattern`, and `--role-arn` parameters.
The following sample command creates a rule named `EnabledS3SourceRule`.
```
aws events put-rule --name "EnabledS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"Object Created\"],\"detail\":{\"bucket\":{\"name\":[\"amzn-s3-demo-source-bucket\"]}}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
```
1. To add CodePipeline as a target, call the **put-targets** command and include the `--rule` and `--targets` parameters.
The following command specifies that for the rule named `EnabledS3SourceRule`, the target `Id` is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an example `ARN` for the pipeline. The pipeline starts when something changes in the repository.
```
aws events put-targets --rule EnabledS3SourceRule --targets Id=codepipeline-AppPipeline,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
```
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. Run the **get-pipeline** command to copy the pipeline structure into a JSON file. For example, for a pipeline named `MyFirstPipeline`, run the following command:
```
aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json
```
This command returns nothing, but the file you created should appear in the directory where you ran the command.
1. Open the JSON file in any plain-text editor and edit the source stage by changing the `PollForSourceChanges` parameter for a bucket named `amzn-s3-demo-source-bucket` to `false`, as shown in this example.
**Why am I making this change?** Setting this parameter to `false` turns off periodic checks so you can use event-based change detection only.
```
"configuration": {
"S3Bucket": "amzn-s3-demo-source-bucket",
"PollForSourceChanges": "false",
"S3ObjectKey": "index.zip"
},
```
1. If you are working with the pipeline structure retrieved using the **get-pipeline** command, you must remove the `metadata` lines from the JSON file. Otherwise, the **update-pipeline** command cannot use it. Remove the `"metadata": { }` lines and the `"created"`, `"pipelineARN"`, and `"updated"` fields.
For example, remove the following lines from the structure:
```
"metadata": {
"pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name",
"created": "date",
"updated": "date"
},
```
Save the file.
1. To apply your changes, run the **update-pipeline** command, specifying the pipeline JSON file:
**Important**
Be sure to include `file://` before the file name. It is required in this command.
```
aws codepipeline update-pipeline --cli-input-json file://pipeline.json
```
This command returns the entire structure of the edited pipeline.
**Note**
The **update-pipeline** command stops the pipeline. If a revision is being run through the pipeline when you run the **update-pipeline** command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the **start-pipeline-execution** command to manually start your pipeline.
# Create pipelines with an S3 source enabled for events (CloudFormation template)
This procedure is for a pipeline where the source bucket has events enabled.
Use these steps to create a pipeline with an Amazon S3 source for event-based change detection.
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then add the following resources to your template:
+ EventBridge rule and IAM role to allow this event to start your pipeline.
If you use CloudFormation to create and manage your pipelines, your template includes content like the following.
**Note**
The `Configuration` property in the source stage called `PollForSourceChanges`. If your template doesn't include that property, then `PollForSourceChanges` is set to `true` by default.
------
#### [ YAML ]
```
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
-
Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref S3SourceObjectKey
PollForSourceChanges: true
RunOrder: 1
...
```
------
#### [ JSON ]
```
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"RoleArn": {
"Fn::GetAtt": ["CodePipelineServiceRole", "Arn"]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": true
},
"RunOrder": 1
}
]
},
...
```
------
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. In the template, under `Resources`, use the `AWS::IAM::Role` CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:
+ The first policy allows the role to be assumed.
+ The second policy provides permissions to start the pipeline.
**Why am I making this change?** Adding `AWS::IAM::Role` resource enables CloudFormation to create permissions for EventBridge. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
...
```
------
#### [ JSON ]
```
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
...
```
------
1. Use the `AWS::Events::Rule` CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors creation or deletion of objects in your Amazon S3 source bucket. In addition, include a target of your pipeline. When an object is created, this rule invokes `StartPipelineExecution` on your target pipeline.
**Why am I making this change?** Adding the `AWS::Events::Rule` resource enables CloudFormation to create the event. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRule:
Type: AWS::Events::Rule
Properties:
EventBusName: default
EventPattern:
source:
- aws.s3
detail-type:
- Object Created
detail:
bucket:
name:
- !Ref SourceBucket
Name: EnabledS3SourceRule
State: ENABLED
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
...
```
------
#### [ JSON ]
```
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventBusName": "default",
"EventPattern": {
"source": [
"aws.s3"
],
"detail-type": [
"Object Created"
],
"detail": {
"bucket": {
"name": [
"s3-pipeline-source-fra-bucket"
]
}
}
},
"Name": "EnabledS3SourceRule",
"State": "ENABLED",
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
}
},
...
```
------
1. Save your updated template to your local computer, and open the CloudFormation console.
1. Choose your stack, and then choose **Create Change Set for Current Stack**.
1. Upload your updated template, and then view the changes listed in CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.
1. Choose **Execute**.
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
+ In the template, change `PollForSourceChanges` to `false`. If you did not include `PollForSourceChanges` in your pipeline definition, add it and set it to `false`.
**Why am I making this change?** Changing `PollForSourceChanges` to `false` turns off periodic checks so you can use event-based change detection only.
------
#### [ YAML ]
```
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
```
------
#### [ JSON ]
```
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
```
------
**Example**
When you use CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.
Do not stop here. Although your pipeline is created, you must create a second CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.
```
Parameters:
SourceObjectKey:
Description: 'S3 source artifact'
Type: String
Default: SampleApp_Linux.zip
ApplicationName:
Description: 'CodeDeploy application name'
Type: String
Default: DemoApplication
BetaFleet:
Description: 'Fleet configured in CodeDeploy'
Type: String
Default: DemoFleet
Resources:
SourceBucket:
Type: AWS::S3::Bucket
Properties:
NotificationConfiguration:
EventBridgeConfiguration:
EventBridgeEnabled: true
VersioningConfiguration:
Status: Enabled
CodePipelineArtifactStoreBucket:
Type: AWS::S3::Bucket
CodePipelineArtifactStoreBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref CodePipelineArtifactStoreBucket
PolicyDocument:
Version: 2012-10-17
Statement:
-
Sid: DenyUnEncryptedObjectUploads
Effect: Deny
Principal: '*'
Action: s3:PutObject
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
StringNotEquals:
s3:x-amz-server-side-encryption: aws:kms
-
Sid: DenyInsecureConnections
Effect: Deny
Principal: '*'
Action: s3:*
Resource: !Sub ${CodePipelineArtifactStoreBucket.Arn}/*
Condition:
Bool:
aws:SecureTransport: false
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: AWS-CodePipeline-Service-3
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- devicefarm:ListProjects
- devicefarm:ListDevicePools
- devicefarm:GetRun
- devicefarm:GetUpload
- devicefarm:CreateUpload
- devicefarm:ScheduleRun
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:ListFunctions
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- iam:PassRole
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- s3:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource: 'resource_ARN'
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: s3-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
-
Name: Beta
Actions:
-
Name: BetaAction
InputArtifacts:
- Name: SourceOutput
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CodeDeploy
Configuration:
ApplicationName: !Ref ApplicationName
DeploymentGroupName: !Ref BetaFleet
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref CodePipelineArtifactStoreBucket
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
EventRule:
Type: AWS::Events::Rule
Properties:
EventBusName: default
EventPattern:
source:
- aws.s3
detail-type:
- Object Created
detail:
bucket:
name:
- !Ref SourceBucket
Name: EnabledS3SourceRule
State: ENABLED
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
```
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"appconfig:StartDeployment",
"appconfig:StopDeployment",
"appconfig:GetDeployment"
],
"Resource": [
"arn:aws:appconfig:*:111122223333:application/[[Application]]",
"arn:aws:appconfig:*:111122223333:application/[[Application]]/*",
"arn:aws:appconfig:*:111122223333:deploymentstrategy/*"
],
"Effect": "Allow"
}
]
}
```