# Use AWS CodeBuild with Amazon Virtual Private Cloud
Typically, AWS CodeBuild cannot access resources in a VPC. To enable access, you must provide additional VPC-specific configuration information in your CodeBuild project configuration. This includes the VPC ID, the VPC subnet IDs, and the VPC security group IDs. VPC-enabled builds can then access resources inside your VPC. For more information about setting up a VPC in Amazon VPC, see the [Amazon VPC User Guide](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Introduction.html).
**Topics**
+ [
## Use cases
](#use-cases)
+ [
## Best practices for VPCs
](#best-practices-for-vpcs)
+ [
## Limitations of VPCs
](#vpc-limitations)
+ [
# Allow Amazon VPC access in your CodeBuild projects
](enabling-vpc-access-in-projects.md)
+ [
# Troubleshoot your VPC setup
](troubleshooting-vpc.md)
+ [
# Use VPC endpoints
](use-vpc-endpoints-with-codebuild.md)
+ [
# Use AWS CodeBuild with a managed proxy server
](run-codebuild-in-managed-proxy-server.md)
+ [
# Use AWS CodeBuild with a proxy server
](use-proxy-server.md)
+ [
# CloudFormation VPC template
](cloudformation-vpc-template.md)
## Use cases
VPC connectivity from AWS CodeBuild builds makes it possible to:
+ Run integration tests from your build against data in an Amazon RDS database that's isolated on a private subnet.
+ Query data in an Amazon ElastiCache cluster directly from tests.
+ Interact with internal web services hosted on Amazon EC2, Amazon ECS, or services that use internal Elastic Load Balancing.
+ Retrieve dependencies from self-hosted, internal artifact repositories, such as PyPI for Python, Maven for Java, and npm for Node.js.
+ Access objects in an S3 bucket configured to allow access through an Amazon VPC endpoint only.
+ Query external web services that require fixed IP addresses through the Elastic IP address of the NAT gateway or NAT instance associated with your subnet.
Your builds can access any resource that's hosted in your VPC.
## Best practices for VPCs
Use this checklist when you set up a VPC to work with CodeBuild.
+ Set up your VPC with public and private subnets, and a NAT gateway. The NAT gateway must reside in a public subnet. For more information, see [VPC with public and private subnets (NAT)](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario2.html) in the *Amazon VPC User Guide*.
**Important**
You need a NAT gateway or NAT instance to use CodeBuild with your VPC so that CodeBuild can reach public endpoints (for example, to run CLI commands when running builds). You cannot use the internet gateway instead of a NAT gateway or a NAT instance because CodeBuild does not support assigning Elastic IP addresses to the network interfaces that it creates, and auto-assigning a public IP address is not supported by Amazon EC2 for any network interfaces created outside of Amazon EC2 instance launches.
+ Include multiple Availability Zones with your VPC.
+ Make sure that your security groups have no inbound (ingress) traffic allowed to your builds. CodeBuild does not have specific requirements for outbound traffic, but you must allow access to any Internet resources required for your build, such as GitHub or Amazon S3.
For more information, see [Security groups rules](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#SecurityGroupRules) in the *Amazon VPC User Guide*.
+ Set up separate subnets for your builds.
+ When you set up your CodeBuild projects to access your VPC, choose private subnets only.
For more information about setting up a VPC in Amazon VPC, see the [Amazon VPC User Guide](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Introduction.html).
For more information about using CloudFormation to configure a VPC to use the CodeBuild VPC feature, see the [CloudFormation VPC template](cloudformation-vpc-template.md).
## Limitations of VPCs
+ VPC connectivity from CodeBuild is not supported for shared VPCs.
# Allow Amazon VPC access in your CodeBuild projects
Include these settings in your VPC configuration:
+ For **VPC ID**, choose the VPC ID that CodeBuild uses.
+ For **Subnets**, choose a private subnet with NAT translation that includes or has routes to the resources used by CodeBuild.
+ For **Security Groups**, choose the security groups that CodeBuild uses to allow access to resources in the VPCs.
To use the console to create a build project, see [Create a build project (console)](create-project.md#create-project-console). When you create or change your CodeBuild project, in **VPC**, choose your VPC ID, subnets, and security groups.
To use the AWS CLI to create a build project, see [Create a build project (AWS CLI)](create-project.md#create-project-cli). If you are using the AWS CLI with CodeBuild, the service role used by CodeBuild to interact with services on behalf of the IAM user must have a policy attached. For information, see [Allow CodeBuild access to AWS services required to create a VPC network interface](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-create-vpc-network-interface).
The *vpcConfig* object should include your *vpcId*, *securityGroupIds*, and *subnets*.
+ *vpcId*: Required. The VPC ID that CodeBuild uses. Run this command to get a list of all Amazon VPC IDs in your Region:
```
aws ec2 describe-vpcs
```
+ *subnets*: Required. The subnet IDs that include resources used by CodeBuild. Run this command obtain these IDs:
```
aws ec2 describe-subnets --filters "Name=vpc-id,Values=" --region us-east-1
```
**Note**
Replace `us-east-1` with your Region.
+ *securityGroupIds*: Required. The security group IDs used by CodeBuild to allow access to resources in the VPCs. Run this command to obtain these IDs:
```
aws ec2 describe-security-groups --filters "Name=vpc-id,Values=" --region us-east-1
```
**Note**
Replace `us-east-1` with your Region.
# Troubleshoot your VPC setup
Use the information that appears in the error message to help you identify, diagnose, and address issues.
The following are some guidelines to assist you when troubleshooting a common CodeBuild VPC error: `Build does not have internet connectivity. Please check subnet network configuration`.
1. [Make sure that your internet gateway is attached to VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html#Add_IGW_Attach_Gateway).
1. [Make sure that the route table for your public subnet points to the internet gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Route_Tables.html#route-tables-internet-gateway).
1. [Make sure that your network ACLs allow traffic to flow](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#SecurityGroupRules).
1. [Make sure that your security groups allow traffic to flow](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#SecurityGroupRules).
1. [Troubleshoot your NAT gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC-nat-gateway.html#nat-gateway-troubleshooting).
1. [Make sure that the route table for private subnets points to the NAT gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Route_Tables.html#route-tables-nat).
1. Make sure that the service role used by CodeBuild to interact with services on behalf of the IAM user has the permissions in [ this policy](https://docs.aws.amazon.com/codebuild/latest/userguide/auth-and-access-control-iam-identity-based-access-control.html#customer-managed-policies-example-create-vpc-network-interface). For more information, see [Allow CodeBuild to interact with other AWS services](setting-up-service-role.md).
If CodeBuild is missing permissions, you might receive an error that says, `Unexpected EC2 error: UnauthorizedOperation`. This error can occur if CodeBuild does not have the Amazon EC2 permissions required to work with a VPC.
# Use VPC endpoints
You can improve the security of your builds by configuring AWS CodeBuild to use an interface VPC endpoint. Interface endpoints are powered by PrivateLink, a technology that you can use to privately access Amazon EC2 and CodeBuild by using private IP addresses. PrivateLink restricts all network traffic between your managed instances, CodeBuild, and Amazon EC2 to the Amazon network. (Managed instances don't have access to the internet.) Also, you don't need an internet gateway, NAT device, or virtual private gateway. You are not required to configure PrivateLink, but it's recommended. For more information about PrivateLink and VPC endpoints, see [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html).
## Before you create VPC endpoints
Before you configure VPC endpoints for AWS CodeBuild, be aware of the following restrictions and limitations.
**Note**
Use a [NAT gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html) if you want to use CodeBuild with AWS services that do not support Amazon VPC PrivateLink connections.
+ VPC endpoints support Amazon-provided DNS through Amazon Route 53 only. If you want to use your own DNS, you can use conditional DNS forwarding. For more information, see [DHCP option sets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_DHCP_Options.html) in the *Amazon VPC User Guide*.
+ VPC endpoints currently do not support cross-Region requests. Make sure that you create your endpoint in the same AWS Region as any S3 buckets that store your build input and output. You can use the Amazon S3 console or the [get-bucket-location](https://docs.aws.amazon.com/cli/latest/reference/s3api/get-bucket-location.html) command to find the location of your bucket. Use a Region-specific Amazon S3 endpoint to access your bucket (for example, `.s3-us-west-2.amazonaws.com`). For more information about Region-specific endpoints for Amazon S3, see [Amazon Simple Storage Service](https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) in the *Amazon Web Services General Reference*. If you use the AWS CLI to make requests to Amazon S3, set your default Region to the same Region where your bucket was created, or use the `--region` parameter in your requests.
## Create VPC endpoints for CodeBuild
Follow the instructions in [Creating an interface endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) to create the endpoint `com.amazonaws.region.codebuild`. This is a VPC endpoint for AWS CodeBuild.
![\[The VPC endpoint configuration.\]](http://docs.aws.amazon.com/codebuild/latest/userguide/images/vpc-endpoint.png)
*region* represents the region identifier for an AWS Region supported by CodeBuild, such as `us-east-2` for the US East (Ohio) Region. For a list of supported AWS Regions, see [CodeBuild](https://docs.aws.amazon.com/general/latest/gr/rande.html#codebuild_region) in the * AWS General Reference*. The endpoint is prepopulated with the Region you specified when you signed in to AWS. If you change your Region, the VPC endpoint is updated accordingly.
## Create a VPC endpoint policy for CodeBuild
You can create a policy for Amazon VPC endpoints for AWS CodeBuild in which you can specify:
+ The principal that can perform actions.
+ The actions that can be performed.
+ The resources that can have actions performed on them.
The following example policy specifies that all principals can only start and view builds for the `project-name` project.
```
{
"Statement": [
{
"Action": [
"codebuild:ListBuildsForProject",
"codebuild:StartBuild",
"codebuild:BatchGetBuilds"
],
"Effect": "Allow",
"Resource": "arn:aws:codebuild:region-ID:account-ID:project/project-name",
"Principal": "*"
}
]
}
```
For more information, see [Controlling access to services with VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-access.html) in the *Amazon VPC User Guide*.
# Use AWS CodeBuild with a managed proxy server
To run AWS CodeBuild reserved capacity fleets in a managed proxy server, you must configure the proxy server to allow or deny traffic to and from external sites using proxy rules. Note that running reserved capacity fleets in a managed proxy server is not supported for VPC, Windows, or MacOS.
**Important**
There are additional costs based on the duration that a proxy configuration is present in the fleet. For more information, see [https://aws.amazon.com/codebuild/pricing/](https://aws.amazon.com/codebuild/pricing/).
**Topics**
+ [
## Configure a managed proxy configuration for reserved capacity fleets
](#run-codebuild-in-managed-proxy-server-configure)
+ [
## Run a CodeBuild reserved capacity fleet
](#use-managed-server-run-acb-fleet)
## Configure a managed proxy configuration for reserved capacity fleets
To configure a managed proxy server for your reserved capacity fleet, you must enable this feature when creating your fleet in your console or using the AWS CLI. There are several properties which you need to define:
**Define proxy configurations - optional**
Proxy configurations that apply network access control to your reserved capacity instances.
**Default behavior**
Defines the behavior of outgoing traffic.
**Allow**
Allows outgoing traffic to all destinations by default.
**Deny**
Denies outgoing traffic to all destinations by default.
**Proxy rules**
Specifies destination domains to restrict network access control to.
To define proxy configurations in your console, see [Create a reserved capacity fleet](fleets.md#fleets.how-to) for instructions. To define proxy configurations using the AWS CLI, you can do so by modifying the following JSON syntax and saving your results:
```
"proxyConfiguration": {
"defaultBehavior": "ALLOW_ALL" | "DENY_ALL",
"orderedProxyRules": [
{
"type": "DOMAIN" | "IP",
"effect": "ALLOW" | "DENY",
"entities": [
"destination"
]
}
]
}
```
Your JSON file may look similar to the following:
```
"proxyConfiguration": {
"defaultBehavior": "DENY_ALL",
"orderedProxyRules": [
{
"type": "DOMAIN",
"effect": "ALLOW",
"entities": [
"github.com"
]
}
]
}
```
## Run a CodeBuild reserved capacity fleet
When running AWS CodeBuild reserved capacity fleets with your managed proxy server, CodeBuild will automatically set its `HTTP_PROXY` and `HTTPS_PROXY` environment variables with the managed proxy addresses. If your dependency software has its own configuration and does not adhere to the environment variables, you can refer to these values and update your software configuration in your build commands to properly route your build traffic through the managed proxy. For more information, see [Create a build project in AWS CodeBuild](create-project.md) and [Change build project settings in AWS CodeBuild](change-project.md).
# Use AWS CodeBuild with a proxy server
You can use AWS CodeBuild with a proxy server to regulate HTTP and HTTPS traffic to and from the internet. To run CodeBuild with a proxy server, you install a proxy server in a public subnet and CodeBuild in a private subnet in a VPC.
There are two primary use cases for running CodeBuild in a proxy server:
+ It eliminates the use of a NAT gateway or NAT instance in your VPC.
+ It lets you specify the URLs that instances in the proxy server can access and the URLs to which the proxy server denies access.
You can use CodeBuild with two types of proxy servers. For both, the proxy server runs in a public subnet and CodeBuild runs in a private subnet.
+ **Explicit proxy**: If you use an explicit proxy server, you must configure `NO_PROXY`, `HTTP_PROXY`, and `HTTPS_PROXY` environment variables in CodeBuild at the project level. For more information, see [Change build project settings in AWS CodeBuild](change-project.md) and [Create a build project in AWS CodeBuild](create-project.md).
+ **Transparent proxy**: If you use a transparent proxy server, no special configuration is required.
**Topics**
+ [
# Set up components required to run CodeBuild in a proxy server
](use-proxy-server-transparent-components.md)
+ [
# Run CodeBuild in an explicit proxy server
](run-codebuild-in-explicit-proxy-server.md)
+ [
# Run CodeBuild in a transparent proxy server
](run-codebuild-in-transparent-proxy-server.md)
+ [
# Run a package manager and other tools in a proxy server
](use-proxy-server-tools.md)
# Set up components required to run CodeBuild in a proxy server
You need these components to run AWS CodeBuild in a transparent or explicit proxy server:
+ A VPC.
+ One public subnet in your VPC for the proxy server.
+ One private subnet in your VPC for CodeBuild.
+ An internet gateway that allows communcation between the VPC and the internet.
The following diagram shows how the components interact.
![\[The diagram shows how the components interact.\]](http://docs.aws.amazon.com/codebuild/latest/userguide/images/codebuild-proxy-transparent.png)
## Set up a VPC, subnets, and a network gateway
The following steps are required to run AWS CodeBuild in a transparent or explicit proxy server.
1. Create a VPC. For information, see [Creating a VPC](https://docs.aws.amazon.com/vpc/latest/userguide/working-with-vpcs.html#Create-VPC) in the *Amazon VPC User Guide*.
1. Create two subnets in your VPC. One is a public subnet named `Public Subnet` in which your proxy server runs. The other is a private subnet named `Private Subnet` in which CodeBuild runs.
For information, see [Creating a subnet in your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/working-with-vpcs.html#AddaSubnet).
1. Create and attach an internet gateway to your VPC. For more information, see [Creating and attaching an internet gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html#Add_IGW_Attach_Gateway).
1. Add a rule to the default route table that routes outgoing traffic from the VPC (0.0.0.0/0) to the internet gateway. For information, see [Adding and removing routes from a route table](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Route_Tables.html#AddRemoveRoutes).
1. Add a rule to the default security group of your VPC that allows ingress SSH traffic (TCP 22) from your VPC (0.0.0.0/0).
1. Follow the instructions in [Launching an instance using the launch instance wizard](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/launching-instance.html) in the *Amazon EC2 User Guide* to launch an Amazon Linux instance. When you run the wizard, choose the following options:
+ In **Choose an Instance Type**, choose an Amazon Linux Amazon Machine Image (AMI).
+ In **Subnet**, choose the public subnet you created earlier in this topic. If you used the suggested name, it is **Public Subnet**.
+ In **Auto-assign Public IP**, choose **Enable**.
+ On the **Configure Security Group** page, for **Assign a security group**, choose **Select an existing security group**. Next, choose the default security group.
+ After you choose **Launch**, choose an existing key pair or create one.
Choose the default settings for all other options.
1. After your EC2 instance is running, disable source/destination checks. For information, see [Disabling Source/Destination checks](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck) in the *Amazon VPC User Guide*.
1. Create a route table in your VPC. Add a rule to the route table that routes traffic destined for the internet to your proxy server. Associate this route table with your private subnet. This is required so that outbound requests from instances in your private subnet, where CodeBuild runs, are always routed through the proxy server.
## Install and configure a proxy server
There are many proxy servers from which to choose. An open-source proxy server, Squid, is used here to demonstrate how AWS CodeBuild runs in a proxy server. You can apply the same concepts to other proxy servers.
To install Squid, use a yum repo by running the following commands:
```
sudo yum update -y
sudo yum install -y squid
```
After you install Squid, edit its `squid.conf` file using the instructions later in this topic.
## Configure Squid for HTTPS traffic
For HTTPS, the HTTP traffic is encapsulated in a Transport Layer Security (TLS) connection. Squid uses a feature called [SslPeekAndSplice](https://wiki.squid-cache.org/Features/SslPeekAndSplice) to retrieve the Server Name Indication (SNI) from the TLS initiation that contains the requested internet host. This is required so Squid does not need to unencrypt HTTPS traffic. To enable SslPeekAndSplice, Squid requires a certificate. Create this certificate using OpenSSL:
```
sudo mkdir /etc/squid/ssl
cd /etc/squid/ssl
sudo openssl genrsa -out squid.key 2048
sudo openssl req -new -key squid.key -out squid.csr -subj "/C=XX/ST=XX/L=squid/O=squid/CN=squid"
sudo openssl x509 -req -days 3650 -in squid.csr -signkey squid.key -out squid.crt
sudo cat squid.key squid.crt | sudo tee squid.pem
```
**Note**
For HTTP, Squid does not require configuration. From all HTTP/1.1 request messages, it can retrieve the host header field, which specifies the internet host that is being requested.
# Run CodeBuild in an explicit proxy server
To run AWS CodeBuild in an explicit proxy server, you must configure the proxy server to allow or deny traffic to and from external sites, and then configure the `HTTP_PROXY` and `HTTPS_PROXY` environment variables.
**Topics**
+ [
## Configure Squid as an explicit proxy server
](#use-proxy-server-explicit-squid-configure)
+ [
## Create a CodeBuild project
](#use-proxy-server-explicit-create-acb-project)
+ [
## Explicit proxy server sample `squid.conf` file
](#use-proxy-server-explicit-sample-squid-conf)
## Configure Squid as an explicit proxy server
To configure the Squid proxy server to be explicit, you must make the following modifications to its `/etc/squid/squid.conf` file:
+ Remove the following default access control list (ACL) rules.
```
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.0.0/16
acl localnet src fc00::/7
acl localnet src fe80::/10
```
Add the following in place of the default ACL rules you removed. The first line allows requests from your VPC. The next two lines grant your proxy server access to destination URLs that might be used by AWS CodeBuild. Edit the regular expression in the last line to specify S3 buckets or a CodeCommit repository in an AWS Region. For example:
+ If your source is Amazon S3, use the command **acl download\$1src dstdom\$1regex .\$1s3\$1.us-west-1\$1.amazonaws\$1.com**to grant access to S3 buckets in the `us-west-1` Region.
+ If your source is AWS CodeCommit, use `git-codecommit..amazonaws.com` to add an AWS Region to an allow list.
```
acl localnet src 10.1.0.0/16 #Only allow requests from within the VPC
acl allowed_sites dstdomain .github.com #Allows to download source from GitHub
acl allowed_sites dstdomain .bitbucket.com #Allows to download source from Bitbucket
acl download_src dstdom_regex .*\.amazonaws\.com #Allows to download source from Amazon S3 or CodeCommit
```
+ Replace `http_access allow localnet` with the following:
```
http_access allow localnet allowed_sites
http_access allow localnet download_src
```
+ If you want your build to upload logs and artifacts, do one of the following:
1. Before the `http_access deny all` statement, insert the following statements. They allow CodeBuild to access CloudWatch and Amazon S3. Access to CloudWatch is required so that CodeBuild can create CloudWatch logs. Access to Amazon S3 is required for uploading artifacts and Amazon S3 caching.
+
```
https_port 3130 cert=/etc/squid/ssl/squid.pem ssl-bump intercept
acl SSL_port port 443
http_access allow SSL_port
acl allowed_https_sites ssl::server_name .amazonaws.com
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump peek step2 allowed_https_sites
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate step2 all
```
+ After you save `squid.conf`, run the following command:
```
sudo iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 3130
sudo service squid restart
```
1. Add `proxy` to your buildspec file. For more information, see [Buildspec syntax](build-spec-ref.md#build-spec-ref-syntax).
```
version: 0.2
proxy:
upload-artifacts: yes
logs: yes
phases:
build:
commands:
- command
```
**Note**
If you receive a RequestError timeout error, see [RequestError timeout error when running CodeBuild in a proxy server](troubleshooting.md#code-request-timeout-error).
For more information, see [Explicit proxy server sample `squid.conf` file](#use-proxy-server-explicit-sample-squid-conf) later in this topic.
## Create a CodeBuild project
To run AWS CodeBuild with your explicit proxy server, set its `HTTP_PROXY` and `HTTPS_PROXY` environment variables with the private IP address of the EC2 instance you created for your proxy server and port 3128 at the project level. The private IP address looks like `http://your-ec2-private-ip-address:3128`. For more information, see [Create a build project in AWS CodeBuild](create-project.md) and [Change build project settings in AWS CodeBuild](change-project.md).
Use the following command to view the Squid proxy access log:
```
sudo tail -f /var/log/squid/access.log
```
## Explicit proxy server sample `squid.conf` file
The following is an example of a `squid.conf` file that is configured for an explicit proxy server.
```
acl localnet src 10.0.0.0/16 #Only allow requests from within the VPC
# add all URLS to be whitelisted for download source and commands to be run in build environment
acl allowed_sites dstdomain .github.com #Allows to download source from github
acl allowed_sites dstdomain .bitbucket.com #Allows to download source from bitbucket
acl allowed_sites dstdomain ppa.launchpad.net #Allows to run apt-get in build environment
acl download_src dstdom_regex .*\.amazonaws\.com #Allows to download source from S3 or CodeCommit
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet allowed_sites
http_access allow localnet download_src
http_access allow localhost
# Add this for CodeBuild to access CWL end point, caching and upload artifacts S3 bucket end point
https_port 3130 cert=/etc/squid/ssl/squid.pem ssl-bump intercept
acl SSL_port port 443
http_access allow SSL_port
acl allowed_https_sites ssl::server_name .amazonaws.com
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump peek step2 allowed_https_sites
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate step2 all
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 3128
# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
```
# Run CodeBuild in a transparent proxy server
To run AWS CodeBuild in a transparent proxy server, you must configure the proxy server with access to the websites and domains it interacts with.
**Topics**
+ [
## Configure Squid as a transparent proxy server
](#use-proxy-server-transparent-squid-configure)
+ [
## Create a CodeBuild project
](#use-proxy-server-transparent-create-acb-project)
## Configure Squid as a transparent proxy server
To configure a proxy server to be transparent, you must grant it access to the domains and websites you want it to access. To run AWS CodeBuild with a transparent proxy server, you must grant it access to `amazonaws.com`. You must also grant access to other websites CodeBuild uses. These vary, depending on how you create your CodeBuild projects. Example websites are those for repositories such as GitHub, Bitbucket, Yum, and Maven. To grant Squid access to specific domains and websites, use a command similar to the following to update the `squid.conf` file. This sample command grants access to `amazonaws.com`, `github.com`, and `bitbucket.com`. You can edit this sample to grant access to other websites.
```
cat | sudo tee /etc/squid/squid.conf ≪EOF
visible_hostname squid
#Handling HTTP requests
http_port 3129 intercept
acl allowed_http_sites dstdomain .amazonaws.com
#acl allowed_http_sites dstdomain domain_name [uncomment this line to add another domain]
http_access allow allowed_http_sites
#Handling HTTPS requests
https_port 3130 cert=/etc/squid/ssl/squid.pem ssl-bump intercept
acl SSL_port port 443
http_access allow SSL_port
acl allowed_https_sites ssl::server_name .amazonaws.com
acl allowed_https_sites ssl::server_name .github.com
acl allowed_https_sites ssl::server_name .bitbucket.com
#acl allowed_https_sites ssl::server_name [uncomment this line to add another website]
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump peek step2 allowed_https_sites
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate step2 all
http_access deny all
EOF
```
Incoming requests from instances in the private subnet must redirect to the Squid ports. Squid listens on port 3129 for HTTP traffic (instead of 80) and 3130 for HTTPS traffic (instead of 443). Use the **iptables** command to route traffic:
```
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 3129
sudo iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 3130
sudo service iptables save
sudo service squid start
```
## Create a CodeBuild project
After you configure your proxy server, you can use it with AWS CodeBuild in a private subnet without more configuration. Every HTTP and HTTPS request goes through the public proxy server. Use the following command to view the Squid proxy access log:
```
sudo tail -f /var/log/squid/access.log
```
# Run a package manager and other tools in a proxy server
Use the following procedures to run a package manager and other tools in a proxy server.
**To run a tool, such as a package manager, in a proxy server**
1. Add the tool to the allow list in your proxy server by adding statements to your `squid.conf` file.
1. Add a line to your buildspec file that points to the private endpoint of your proxy server.
The following examples demonstrate how to do this for `apt-get`, `curl`, and `maven`. If you use a different tool, the same principles apply. Add it to an allow list in the `squid.conf` file and add a command to your buildspec file to make CodeBuild aware of your proxy server's endpoint.
**To run `apt-get` in a proxy server**
1. Add the following statements to your `squid.conf` file to add `apt-get` to an allow list in your proxy server. The first three lines allow `apt-get` to run in the build environment.
```
acl allowed_sites dstdomain ppa.launchpad.net # Required for apt-get to run in the build environment
acl apt_get dstdom_regex .*\.launchpad.net # Required for CodeBuild to run apt-get in the build environment
acl apt_get dstdom_regex .*\.ubuntu.com # Required for CodeBuild to run apt-get in the build environment
http_access allow localnet allowed_sites
http_access allow localnet apt_get
```
1. Add the following statement in your buildspec file so that `apt-get` commands look for the proxy configuration in `/etc/apt/apt.conf.d/00proxy`.
```
echo 'Acquire::http::Proxy "http://:3128"; Acquire::https::Proxy "http://:3128"; Acquire::ftp::Proxy "http://:3128";' > /etc/apt/apt.conf.d/00proxy
```
**To run `curl` in a proxy server**
1. Add the following to your `squid.conf` file to add `curl` to an allow list in your build environment.
```
acl allowed_sites dstdomain ppa.launchpad.net # Required to run apt-get in the build environment
acl allowed_sites dstdomain google.com # Required for access to a webiste. This example uses www.google.com.
http_access allow localnet allowed_sites
http_access allow localnet apt_get
```
1. Add the following statement in your buildspec file so `curl` uses the private proxy server to access the website you added to the `squid.conf`. In this example, the website is `google.com`.
```
curl -x :3128 https://www.google.com
```
**To run `maven` in a proxy server**
1. Add the following to your `squid.conf` file to add `maven` to an allow list in your build environment.
```
acl allowed_sites dstdomain ppa.launchpad.net # Required to run apt-get in the build environment
acl maven dstdom_regex .*\.maven.org # Allows access to the maven repository in the build environment
http_access allow localnet allowed_sites
http_access allow localnet maven
```
1. Add the following statement to your buildspec file.
```
maven clean install -DproxySet=true -DproxyHost= -DproxyPort=3128
```
# CloudFormation VPC template
CloudFormation enables you to create and provision AWS infrastructure deployments predictably and repeatedly, by using template files to create and delete a collection of resources together as a single unit (a *stack*). For more information, see the [CloudFormation User Guide](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html).
The following is an CloudFormation YAML template for configuring a VPC to use AWS CodeBuild. This file is also available in [samples.zip](./samples/samples.zip).
```
Description: This template deploys a VPC, with a pair of public and private subnets spread
across two Availability Zones. It deploys an internet gateway, with a default
route on the public subnets. It deploys a pair of NAT gateways (one in each AZ),
and default routes for them in the private subnets.
Parameters:
EnvironmentName:
Description: An environment name that is prefixed to resource names
Type: String
VpcCIDR:
Description: Please enter the IP range (CIDR notation) for this VPC
Type: String
Default: 10.192.0.0/16
PublicSubnet1CIDR:
Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
Type: String
Default: 10.192.10.0/24
PublicSubnet2CIDR:
Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone
Type: String
Default: 10.192.11.0/24
PrivateSubnet1CIDR:
Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone
Type: String
Default: 10.192.20.0/24
PrivateSubnet2CIDR:
Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone
Type: String
Default: 10.192.21.0/24
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcCIDR
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: !Ref EnvironmentName
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: !Ref EnvironmentName
InternetGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref PublicSubnet1CIDR
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Public Subnet (AZ1)
PublicSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [ 1, !GetAZs '' ]
CidrBlock: !Ref PublicSubnet2CIDR
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Public Subnet (AZ2)
PrivateSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref PrivateSubnet1CIDR
MapPublicIpOnLaunch: false
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Private Subnet (AZ1)
PrivateSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [ 1, !GetAZs '' ]
CidrBlock: !Ref PrivateSubnet2CIDR
MapPublicIpOnLaunch: false
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Private Subnet (AZ2)
NatGateway1EIP:
Type: AWS::EC2::EIP
DependsOn: InternetGatewayAttachment
Properties:
Domain: vpc
NatGateway2EIP:
Type: AWS::EC2::EIP
DependsOn: InternetGatewayAttachment
Properties:
Domain: vpc
NatGateway1:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt NatGateway1EIP.AllocationId
SubnetId: !Ref PublicSubnet1
NatGateway2:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt NatGateway2EIP.AllocationId
SubnetId: !Ref PublicSubnet2
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Public Routes
DefaultPublicRoute:
Type: AWS::EC2::Route
DependsOn: InternetGatewayAttachment
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PublicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet1
PublicSubnet2RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet2
PrivateRouteTable1:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Private Routes (AZ1)
DefaultPrivateRoute1:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PrivateRouteTable1
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway1
PrivateSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PrivateRouteTable1
SubnetId: !Ref PrivateSubnet1
PrivateRouteTable2:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Private Routes (AZ2)
DefaultPrivateRoute2:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PrivateRouteTable2
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway2
PrivateSubnet2RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PrivateRouteTable2
SubnetId: !Ref PrivateSubnet2
NoIngressSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "no-ingress-sg"
GroupDescription: "Security group with no ingress rule"
VpcId: !Ref VPC
Outputs:
VPC:
Description: A reference to the created VPC
Value: !Ref VPC
PublicSubnets:
Description: A list of the public subnets
Value: !Join [ ",", [ !Ref PublicSubnet1, !Ref PublicSubnet2 ]]
PrivateSubnets:
Description: A list of the private subnets
Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]]
PublicSubnet1:
Description: A reference to the public subnet in the 1st Availability Zone
Value: !Ref PublicSubnet1
PublicSubnet2:
Description: A reference to the public subnet in the 2nd Availability Zone
Value: !Ref PublicSubnet2
PrivateSubnet1:
Description: A reference to the private subnet in the 1st Availability Zone
Value: !Ref PrivateSubnet1
PrivateSubnet2:
Description: A reference to the private subnet in the 2nd Availability Zone
Value: !Ref PrivateSubnet2
NoIngressSecurityGroup:
Description: Security group with no ingress rule
Value: !Ref NoIngressSecurityGroup
```