Internet access for VPC-connected workflows

When you connect an AWS HealthOmics run to a VPC, the run can only access resources available within that VPC. To give your run access to the public internet or AWS services outside the VPC, you must configure your VPC with the appropriate networking resources.

This topic describes how to set up your VPC to provide internet access and efficient connectivity to AWS services for your VPC-connected runs. For information about connecting runs to a VPC, see Connecting HealthOmics workflows to a VPC.

Setting up a VPC with internet access

To give your VPC-connected runs access to the internet, create a VPC with private subnets that route outbound traffic through a NAT gateway.

This configuration provides:

Supported Regions and Availability Zones

HealthOmics Workflows operates in the following Regions and Availability Zones. When creating your VPC, ensure that your subnets are in one or more of these Availability Zones.

When you create your HealthOmics VPC configuration, specify the private subnets. The runs use the NAT gateway in the public subnet to reach the internet.

VPC endpoints for AWS services

You can configure VPC endpoints to allow runs to access AWS services without traversing the public internet. This improves security and can reduce data transfer costs.

The following table lists commonly used VPC endpoints for HealthOmics runs:

The full list of services that you can access through AWS PrivateLink endpoints can be found in AWS services that integrate with AWS PrivateLink. For detailed endpoint setup instructions, see Access AWS services through AWS PrivateLink in the AWS PrivateLink Guide.

NAT Gateway requirements

For runs requiring public internet access:

Security group configuration

Configure your security groups to allow outbound traffic to the destinations your runs need to access:

The following example shows a security group rule for public internet access:

Route table configuration

Ensure that your private subnets have route table entries that direct internet-bound traffic to a NAT gateway:

For access to on-premises resources, configure routes to a virtual private gateway or gateway.

IAM permissions for AWS services

When your workflow tasks access AWS services such as Amazon Athena, AWS Glue, or Amazon DynamoDB in VPC networking mode, you must add the necessary permissions to the service role that you pass to the StartRun API. Without these permissions, your workflow tasks will fail with AccessDeniedException or UnauthorizedException errors.

If your workflow fails with permission errors, check the CloudWatch Logs logs for your workflow run. Common error messages include AccessDeniedException: You are not authorized to perform: action on the resource (the service role is missing the required IAM permission) or UnrecognizedClientException: The security token included in the request is invalid (the service role trust policy may be misconfigured, or the role ARN passed to StartRun is incorrect).

Common service permissions

The following examples show IAM permissions for commonly used AWS services in VPC mode workflows. Add these permissions to your service role policy based on which services your workflow accesses.

{
  "Effect": "Allow",
  "Action": [
    "athena:StartQueryExecution",
    "athena:GetQueryExecution",
    "athena:GetQueryResults",
    "athena:StopQueryExecution"
  ],
  "Resource": "arn:aws:athena:region:account-id:workgroup/workgroup-name"
}

{
  "Effect": "Allow",
  "Action": [
    "glue:GetDatabase",
    "glue:GetTable",
    "glue:GetPartitions",
    "glue:CreateTable",
    "glue:UpdateTable"
  ],
  "Resource": [
    "arn:aws:glue:region:account-id:catalog",
    "arn:aws:glue:region:account-id:database/database-name",
    "arn:aws:glue:region:account-id:table/database-name/*"
  ]
}

{
  "Effect": "Allow",
  "Action": [
    "dynamodb:GetItem",
    "dynamodb:PutItem",
    "dynamodb:Query",
    "dynamodb:Scan"
  ],
  "Resource": "arn:aws:dynamodb:region:account-id:table/table-name"
}

{
  "Effect": "Allow",
  "Action": [
    "s3tables:GetTableData",
    "s3tables:PutTableData"
  ],
  "Resource": "arn:aws:s3tables:region:account-id:bucket/bucket-name/table/table-id"
}

Testing VPC connectivity

Before running production workflows, validate that your VPC configuration allows connectivity to required external services.

Create a test workflow

Create a simple workflow that tests connectivity to your external service. For example, create a workflow that attempts a TCP connection to a target service endpoint.

Run the test

aws omics start-run \
  --workflow-id test-workflow-id \
  --role-arn role-arn \
  --output-uri s3://bucket-name/test-outputs/ \
  --networking-mode VPC \
  --configuration-name configuration-name \
  --parameters file://test-parameters.json

Verify results

Check the workflow output to confirm successful connectivity:

{
  "connectivity_test.result": "Testing connection to external service...\nSUCCESS: Connection successful!\nTest completed"
}

If the test fails, verify the following:

Examples

Accessing NCBI data with API authentication

This example demonstrates how to access NCBI data using the NCBI Datasets API with authentication.

To get your NCBI API key, visit the NCBI API Keys documentation.

Example workflow definition:

version 1.0
#WORKFLOW DEFINITION

# Meant to be used as integration test for public internet access via VPC tunnel
workflow TestFlow {
    input {
        String ncbi_api_url = "https://api.ncbi.nlm.nih.gov/datasets/v2/gene/accession/NM_021803.4?api_key=<YOUR_API_KEY>"
    }

    call DataProcessTask{
        input:
            ncbi_api_url = ncbi_api_url,
    }

    output {
        File output_file = DataProcessTask.output_file
    }

}

#Task Definitions
task DataProcessTask {
    input {
        String ncbi_api_url
    }

    command <<<
        set -eu
        # Download file from NCBI Datasets API with API key
        curl -fsSL "~{ncbi_api_url}" -o gene_data.json

        # Add data processing task here
        cat gene_data.json > processed_data.json

        # Echo the content to output file
        cat processed_data.json > outfile.txt
    >>>

    output {
        File output_file = "outfile.txt"
    }
}

Key points:

For more information about NCBI API keys and rate limits, see the NCBI Datasets API documentation.

Best practices