本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
创建 VPC 网络
Amazon MWAA 需要 Amazon VPC 和特定的网络组件来支持环境。本指南介绍了为 Amazon MWAA 环境创建 Amazon VPC 网络的不同选项。
注意
Apache Airflow 在低延迟网络环境中效果最好。如果您使用的是将流量路由到其他区域或本地环境的现有 Amazon VPC,我们建议您为 Amazon SQS、CloudWatch、Amazon S3 和 AWS KMS 添加 AWS PrivateLink 端点。有关为 Amazon MWAA 配置 AWS PrivateLink 的更多信息,请参阅创建不可互联网访问的 Amazon VPC 网络。
目录
先决条件
AWS Command Line Interface(AWS CLI)是一种开源工具,让您能够在命令行 Shell 中使用命令与 AWS 服务进行交互。要完成本节中的步骤,您需要以下满足以下条件:
开始前的准备工作
-
环境创建后您无法更改为环境指定的 VPC 网络。
-
您可以为 Amazon VPC 和 Apache Airflow Web 服务器使用私有或公共路由。要查看选项列表,请参阅 Amazon VPC 和 Apache Airflow 访问模式的示例用例。
创建 Amazon VPC 网络的选项
下一节介绍可用于为环境创建 Amazon VPC 网络的选项。
注意
Amazon MWAA 不支持在美国东部(弗吉尼亚州北部)区域中使用 use1-az3
可用区(AZ)。在美国东部(弗吉尼亚州北部)区域中创建用于 Amazon MWAA 的 VPC 时,必须在 AWS CloudFormation(CFN)模板中显式分配 AvailabilityZone
。分配的可用区名称不得映射到 use1-az3
。您可以通过运行以下命令来检索可用区名称与其对应可用区 ID 的详细映射:
aws ec2 describe-availability-zones --region us-east-1
选项一:在 Amazon MWAA 控制台上创建 VPC 网络
下一节显示如何在 Amazon MWAA 控制台上创建 VPC 网络。此选项使用 通过互联网进行公共路由。它可用于具有私有网络或公有网络访问模式的 Apache Airflow Web 服务器。
下图显示了在 Amazon MWAA 控制台上哪里可以找到创建 MWAA VPC 按钮。
选项二:创建可访问互联网的 Amazon VPC 网络
以下 AWS CloudFormation 模板在默认 AWS 区域创建可互联网访问的 Amazon VPC 网络。此选项使用 通过互联网进行公共路由。此模板可用于具有私有网络或公有网络访问模式的 Apache Airflow Web 服务器。
-
复制以下模板的内容并将其作为
cfn-vpc-public-private.yaml
保存在本地中。您也可以使用下载模板。Description: This template deploys a VPC, with a pair of public and private subnets spread across two Availability Zones. It deploys an internet gateway, with a default route on the public subnets. It deploys a pair of NAT gateways (one in each AZ), and default routes for them in the private subnets. Parameters: EnvironmentName: Description: An environment name that is prefixed to resource names Type: String Default: mwaa- VpcCIDR: Description: Please enter the IP range (CIDR notation) for this VPC Type: String Default: 10.192.0.0/16 PublicSubnet1CIDR: Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone Type: String Default: 10.192.10.0/24 PublicSubnet2CIDR: Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone Type: String Default: 10.192.11.0/24 PrivateSubnet1CIDR: Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone Type: String Default: 10.192.20.0/24 PrivateSubnet2CIDR: Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone Type: String Default: 10.192.21.0/24 Resources: VPC: Type: AWS::EC2::VPC Properties: CidrBlock: !Ref VpcCIDR EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Ref EnvironmentName InternetGateway: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: !Ref EnvironmentName InternetGatewayAttachment: Type: AWS::EC2::VPCGatewayAttachment Properties: InternetGatewayId: !Ref InternetGateway VpcId: !Ref VPC PublicSubnet1: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 0, !GetAZs '' ] CidrBlock: !Ref PublicSubnet1CIDR MapPublicIpOnLaunch: true Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Subnet (AZ1) PublicSubnet2: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 1, !GetAZs '' ] CidrBlock: !Ref PublicSubnet2CIDR MapPublicIpOnLaunch: true Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Subnet (AZ2) PrivateSubnet1: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 0, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet1CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Subnet (AZ1) PrivateSubnet2: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 1, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet2CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Subnet (AZ2) NatGateway1EIP: Type: AWS::EC2::EIP DependsOn: InternetGatewayAttachment Properties: Domain: vpc NatGateway2EIP: Type: AWS::EC2::EIP DependsOn: InternetGatewayAttachment Properties: Domain: vpc NatGateway1: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt NatGateway1EIP.AllocationId SubnetId: !Ref PublicSubnet1 NatGateway2: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt NatGateway2EIP.AllocationId SubnetId: !Ref PublicSubnet2 PublicRouteTable: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Routes DefaultPublicRoute: Type: AWS::EC2::Route DependsOn: InternetGatewayAttachment Properties: RouteTableId: !Ref PublicRouteTable DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref InternetGateway PublicSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PublicRouteTable SubnetId: !Ref PublicSubnet1 PublicSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PublicRouteTable SubnetId: !Ref PublicSubnet2 PrivateRouteTable1: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Routes (AZ1) DefaultPrivateRoute1: Type: AWS::EC2::Route Properties: RouteTableId: !Ref PrivateRouteTable1 DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGateway1 PrivateSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PrivateRouteTable1 SubnetId: !Ref PrivateSubnet1 PrivateRouteTable2: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Routes (AZ2) DefaultPrivateRoute2: Type: AWS::EC2::Route Properties: RouteTableId: !Ref PrivateRouteTable2 DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGateway2 PrivateSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PrivateRouteTable2 SubnetId: !Ref PrivateSubnet2 SecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupName: "mwaa-security-group" GroupDescription: "Security group with a self-referencing inbound rule." VpcId: !Ref VPC SecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress Properties: GroupId: !Ref SecurityGroup IpProtocol: "-1" SourceSecurityGroupId: !Ref SecurityGroup Outputs: VPC: Description: A reference to the created VPC Value: !Ref VPC PublicSubnets: Description: A list of the public subnets Value: !Join [ ",", [ !Ref PublicSubnet1, !Ref PublicSubnet2 ]] PrivateSubnets: Description: A list of the private subnets Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]] PublicSubnet1: Description: A reference to the public subnet in the 1st Availability Zone Value: !Ref PublicSubnet1 PublicSubnet2: Description: A reference to the public subnet in the 2nd Availability Zone Value: !Ref PublicSubnet2 PrivateSubnet1: Description: A reference to the private subnet in the 1st Availability Zone Value: !Ref PrivateSubnet1 PrivateSubnet2: Description: A reference to the private subnet in the 2nd Availability Zone Value: !Ref PrivateSubnet2 SecurityGroupIngress: Description: Security group with self-referencing inbound rule Value: !Ref SecurityGroupIngress
-
在命令提示符下,导航到存储
cfn-vpc-public-private.yaml
的目录。例如:cd mwaaproject
-
输入
aws cloudformation create-stack
命令来使用 AWS CLI 创建堆栈。aws cloudformation create-stack --stack-name mwaa-environment --template-body file://cfn-vpc-public-private.yaml
注意
创建 Amazon VPC 基础设施需要大约 30 分钟。
选项三:创建不可互联网访问的 Amazon VPC 网络
以下 AWS CloudFormation 模板将在默认 AWS 区域创建 不可访问互联网的 Amazon VPC 网络。
此选项使用 无法访问互联网的私有路由。此模板只能用于具有私有网络访问模式的 Apache Airflow Web 服务器。它为环境使用的 AWS 服务创建所需的 VPC 端点。
-
复制以下模板的内容并将其作为
cfn-vpc-private.yaml
保存在本地中。您也可以使用下载模板。AWSTemplateFormatVersion: "2010-09-09" Parameters: VpcCIDR: Description: The IP range (CIDR notation) for this VPC Type: String Default: 10.192.0.0/16 PrivateSubnet1CIDR: Description: The IP range (CIDR notation) for the private subnet in the first Availability Zone Type: String Default: 10.192.10.0/24 PrivateSubnet2CIDR: Description: The IP range (CIDR notation) for the private subnet in the second Availability Zone Type: String Default: 10.192.11.0/24 Resources: VPC: Type: AWS::EC2::VPC Properties: CidrBlock: !Ref VpcCIDR EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Ref AWS::StackName RouteTable: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${AWS::StackName}-route-table" PrivateSubnet1: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 0, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet1CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub "${AWS::StackName} Private Subnet (AZ1)" PrivateSubnet2: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 1, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet2CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub "${AWS::StackName} Private Subnet (AZ2)" PrivateSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref RouteTable SubnetId: !Ref PrivateSubnet1 PrivateSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref RouteTable SubnetId: !Ref PrivateSubnet2 S3VpcEndoint: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub "com.amazonaws.${AWS::Region}.s3" VpcEndpointType: Gateway VpcId: !Ref VPC RouteTableIds: - !Ref RouteTable SecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref VPC GroupDescription: Security Group for Amazon MWAA Environments to access VPC endpoints GroupName: !Sub "${AWS::StackName}-mwaa-vpc-endpoints" SecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress Properties: GroupId: !Ref SecurityGroup IpProtocol: "-1" SourceSecurityGroupId: !Ref SecurityGroup SqsVpcEndoint: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub "com.amazonaws.${AWS::Region}.sqs" VpcEndpointType: Interface VpcId: !Ref VPC PrivateDnsEnabled: true SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 SecurityGroupIds: - !Ref SecurityGroup CloudWatchLogsVpcEndoint: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub "com.amazonaws.${AWS::Region}.logs" VpcEndpointType: Interface VpcId: !Ref VPC PrivateDnsEnabled: true SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 SecurityGroupIds: - !Ref SecurityGroup CloudWatchMonitoringVpcEndoint: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub "com.amazonaws.${AWS::Region}.monitoring" VpcEndpointType: Interface VpcId: !Ref VPC PrivateDnsEnabled: true SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 SecurityGroupIds: - !Ref SecurityGroup KmsVpcEndoint: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub "com.amazonaws.${AWS::Region}.kms" VpcEndpointType: Interface VpcId: !Ref VPC PrivateDnsEnabled: true SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 SecurityGroupIds: - !Ref SecurityGroup Outputs: VPC: Description: A reference to the created VPC Value: !Ref VPC MwaaSecurityGroupId: Description: Associates the Security Group to the environment to allow access to the VPC endpoints Value: !Ref SecurityGroup PrivateSubnets: Description: A list of the private subnets Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]] PrivateSubnet1: Description: A reference to the private subnet in the 1st Availability Zone Value: !Ref PrivateSubnet1 PrivateSubnet2: Description: A reference to the private subnet in the 2nd Availability Zone Value: !Ref PrivateSubnet2
-
在命令提示符下,导航到存储
cfn-vpc-private.yml
的目录。例如:cd mwaaproject
-
输入
aws cloudformation create-stack
命令来使用 AWS CLI 创建堆栈。aws cloudformation create-stack --stack-name mwaa-private-environment --template-body file://cfn-vpc-private.yml
注意
创建 Amazon VPC 基础设施需要大约 30 分钟。
-
您需要创建一种机制,以便从计算机访问这些 VPC 端点。要了解更多信息,请参阅 在 Amazon MWAA 上管理对服务特定 Amazon VPC 端点的访问。
注意
您可以在 Amazon MWAA 安全组的 CIDR 中进一步限制出站访问。例如,您可以通过添加自引用出站规则、Amazon S3 的前缀列表和 Amazon VPC 的 CIDR 来限制自身。
接下来做什么?
-
要了解如何创建 Amazon MWAA 环境,请参阅 创建亚马逊MWAA环境。
-
要了解如何使用私有路由创建从计算机到 Amazon VPC 的 VPN 隧道,请参阅 教程:使用 AWS Client VPN 配置私有网络访问权限。