Tutorial: Create a managed compute environment using Amazon EC2 resources - AWS Batch

Tutorial: Create a managed compute environment using Amazon EC2 resources

Complete the following steps to create a mangaed compute environment using Amazon Elastic Compute Cloud (Amazon EC2) resources.

  1. Open the AWS Batch console at https://console.aws.amazon.com/batch/.

  2. From the navigation bar, select the AWS Region to use.

  3. In the navigation pane, choose Compute environments.

  4. Choose Create.

  5. Configure the environment.

    1. For Compute environment configuration, choose Amazon Elastic Compute Cloud (Amazon EC2).

    2. For Orchestration type, choose Managed.

    3. For Name, specify a unique name for your compute environment. The name can contain up to 128 characters in length. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).

    4. For Service role, choose service-linked role that lets the AWS Batch service make calls to the required AWS API operations on your behalf. For example, choose AWSServiceRoleForBatch. For more information, see Service-linked role permissions for AWS Batch.

    5. For Instance role, choose to create a new instance profile or use an existing instance profile that has the required IAM permissions attached. This instance profile allows the Amazon ECS container instances that are created for your compute environment to make calls to the required AWS API operations on your behalf. For more information, see Amazon ECS instance role. If you choose to create a new instance profile, the required role (ecsInstanceRole) is created for you.

    6. (Optional) Expand Tags.

    7. (Optional) For EC2 tags, choose Add tag to add a tag to resources that are launched in the compute environment. Then, enter a Key name and optional Value. Choose Add tag.

    8. (Optional) For Tags, choose Add tag. Then, enter a Key name and optional Value. Choose Add tag.

      For more information, see Tag your AWS Batch resources.

    9. Choose Next page.

  6. In the Instance configuration section:

    1. (Optional) For Enable using Spot instances, turn on Spot. For more information, see Spot Instances.

    2. (Spot only) For Maximum % on-demand price, choose the maximum percentage that a Spot Instance price can be when compared with the On-Demand price for that instance type before instances are launched. For example, if your maximum price is 20%, then the Spot price must be less than 20% of the current On-Demand price for that EC2 instance. You always pay the lowest (market) price and never more than your maximum percentage. If you leave this field empty, the default value is 100% of the On-Demand price.

    3. (Spot only) For Spot fleet role, choose an existing Amazon EC2 Spot Fleet IAM role to apply to your Spot compute environment. If you don't already have an existing Amazon EC2 Spot Fleet IAM role, you must create one first. For more information, see Amazon EC2 spot fleet role.

      Important

      To tag your Spot Instances on creation, your Amazon EC2 Spot Fleet IAM role must use the newer AmazonEC2SpotFleetTaggingRole managed policy. The AmazonEC2SpotFleetRole managed policy doesn't have the required permissions to tag Spot Instances. For more information, see Spot Instances not tagged on creation and Tag your resources.

    4. For Minimum vCPUs, choose the minimum number of vCPUs that your compute environment maintains, regardless of job queue demand.

    5. For Desired vCPUs, choose the number of vCPUs that your compute environment launches with. As your job queue demand increases, AWS Batch can increase the desired number of vCPUs in your compute environment and add EC2 instances, up to the maximum vCPUs. As demand decreases, AWS Batch can decrease the desired number of vCPUs in your compute environment and remove instances, down to the minimum vCPUs.

    6. For Maximum vCPUs, choose the maximum number of vCPUs that your compute environment can scale out to, regardless of job queue demand.

    7. For Allowed instance types, choose the Amazon EC2 instance types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5, c5n, or p3). Or, you can specify specific sizes within a family (such as c5.8xlarge). Metal instance types aren't in the instance families. For example, c5 doesn't include c5.metal. You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.

      Note

      When you create a compute environment, the instance types that you select for the compute environment must share the same architecture. For example, you can't mix x86 and ARM instances in the same compute environment.

      Note

      AWS Batch will scale GPUs based on the required amount in your job queues. To use GPU scheduling, the compute environment must include instance types from the p2, p3, p4, p5, g3, g3s, g4, or g5 families.

      Note

      Currently, optimal uses instance types from the C4, M4, and R4 instance families. In AWS Regions that don't have instance types from those instance families, instance types from the C5, M5, and R5 instance families are used.

    8. Expand Additional configuration.

    9. (Optional) For Placement group, enter a placement group name to group resources in the compute environment.

    10. (Optional) For EC2 key pair, choose a public and private key pair as security credentials when you connect to the instance. For more information about Amazon EC2 key pairs, see Amazon EC2 key pairs and Linux instances.

    11. For Allocation strategy, choose the allocation strategy to use when selecting instance types from the list of allowed instance types. BEST_FIT_PROGRESSIVE is usually the better choice for EC2 On-Demand compute environments, SPOT_CAPACITY_OPTIMIZED, and SPOT_PRICE_CAPACITY_OPTIMIZED for EC2 Spot compute environments. For more information, see Instance type allocation strategies for AWS Batch.

    12. (Optional) For EC2 configuration choose Image type and Image ID override values to provide information for AWS Batch to select Amazon Machine Images (AMIs) for instances in the compute environment. If the Image ID override isn't specified for each Image type, AWS Batch selects a recent Amazon ECS optimized AMI. If no Image type is specified, the default is a Amazon Linux 2 for non-GPU, non AWS Graviton instance.

      Important

      To use a custom AMI, choose the image type and then enter the custom AMI ID in the Image ID override box.

      Amazon Linux 2

      Default for all AWS Graviton-based instance families (for example, C6g, M6g, R6g, and T4g) and can be used for all non-GPU instance types.

      Amazon Linux 2 (GPU)

      Default for all GPU instance families (for example P4 and G4) and can be used for all non AWS Graviton-based instance types.

      Amazon Linux

      Can be used for non-GPU, non AWS Graviton instance families. The standard support for Amazon Linux AMI has ended. For more information, see Amazon Linux AMI.

      Note

      The AMI that you choose for a compute environment must match the architecture of the instance types that you want to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support Arm instances. Amazon ECS vends both x86 and Arm versions of the Amazon ECS optimized Amazon Linux 2 AMI. For more information, see Amazon ECS optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide.

    13. (Optional) For Launch template, select an existing Amazon EC2 launch template to configure your compute resources. The default version of the template is automatically populated. For more information, see Use Amazon EC2 launch templates with AWS Batch.

      Note

      In a launch template, you can specify a custom AMI that you created.

    14. (Optional) For Launch template version, enter $Default, $Latest, or a specific version number to use.

      Note

      Note: If you use either substitution variable ($Default or $Latest), they will apply the current default or latest version number at the time that this configuration is saved. If the default or latest version changes in the future, you must update the information - it won't autamatically update.

      Important

      If the version parameter of the launch template is $Default or $Latest, the default or latest version of the specified launch template is evaluated during an infrastructure update. If a different AMI ID is selected by the default or the latest version of the launch template is selected, that AMI ID is used in the update. For more information, see Updating the AMI ID.

    15. (Optional) For Override launch templates, select an existing Amazon EC2 launch template to use for specific instance types and families.

    16. (Optional) For Override launch template version, enter a specific version number to use, $Default, or $Latest.

      Note

      If you use either the $Default or $Latest variable, AWS Batch will apply the current information at the time that the compute environment is created. If the default or latest version changes in the future, you must update the information through UpdateComputeEnvironment or through the AWS Management Console - AWS Batch.

    17. (Optional) For Target instance types, select the instance type or family that you want to apply the override launch template.

      Note

      If you specify an override launch template, Target instance types is required. For more information, see LaunchTemplateSpecificationOverride.targetInstanceTypes.

      Note

      If the instance type or family that you want to select doesn't appear in this list, review the selections you made in Allowed instance types.

    18. Choose Next page.

  7. In the Network configuration section:

    Important

    Compute resources need access to communicate with the Amazon ECS service endpoint. This can be through an interface VPC endpoint or through your compute resources having public IP addresses.

    For more information about interface VPC endpoints, see Amazon ECS Interface VPC Endpoints (AWS PrivateLink) in the Amazon Elastic Container Service Developer Guide.

    If you do not have an interface VPC endpoint configured and your compute resources do not have public IP addresses, then they must use network address translation (NAT) to provide this access. For more information, see NAT gateways in the Amazon VPC User Guide. For more information, see Tutorial: Create a VPC.

    1. For Virtual Private Cloud (VPC) ID, choose a VPC where to launch your instances.

    2. For Subnets, choose the subnets to use. By default, all subnets within the selected VPC are available.

      Note

      AWS Batch on Amazon EC2 supports Local Zones. For more information, see Local Zones in the Amazon EC2 User Guide and Amazon ECS clusters in Local Zones, Wavelength Zones, and AWS Outposts in the Amazon Elastic Container Service Developer Guide.

    3. (Optional) For Security groups, choose a security group to attach to your instances. By default, the default security group for your VPC is chosen.

      Note

      Note: If you use either substitution variable ($Default or $Latest), they will apply the current default or latest version number at the time that this configuration is saved. If the default or latest version changes in the future, you must update the information - it won't autamatically update.

  8. Choose Next page.

  9. For Review, review the configuration steps. If you need to make changes, choose Edit. When you're finished, choose Create compute environment.