Amazon Athena AWS CMDB connector - Amazon Athena
PrerequisitesParametersDatabases and tablesRequired PermissionsPerformanceLicense informationAdditional resources

Amazon Athena AWS CMDB connector

The Amazon Athena AWS CMDB connector enables Athena to communicate with various AWS services so that you can query them with SQL.

This connector can be registered with Glue Data Catalog as a federated catalog. It supports data access controls defined in Lake Formation at the catalog, database, table, column, row, and tag levels. This connector uses Glue Connections to centralize configuration properties in Glue.

Prerequisites

Parameters

Use the parameters in this section to configure the AWS CMDB connector.

Note

Athena data source connectors created on December 3, 2024 and later use AWS Glue connections.

The parameter names and definitions listed below are for Athena data source connectors created prior to December 3, 2024. These can differ from their corresponding AWS Glue connection properties. Starting December 3, 2024, use the parameters below only when you manually deploy an earlier version of an Athena data source connector.

  • spill_bucket – Specifies the Amazon S3 bucket for data that exceeds Lambda function limits.

  • spill_prefix – (Optional) Defaults to a subfolder in the specified spill_bucket called athena-federation-spill. We recommend that you configure an Amazon S3 storage lifecycle on this location to delete spills older than a predetermined number of days or hours.

  • spill_put_request_headers – (Optional) A JSON encoded map of request headers and values for the Amazon S3 putObject request that is used for spilling (for example, {"x-amz-server-side-encryption" : "AES256"}). For other possible headers, see PutObject in the Amazon Simple Storage Service API Reference.

  • kms_key_id – (Optional) By default, any data that is spilled to Amazon S3 is encrypted using the AES-GCM authenticated encryption mode and a randomly generated key. To have your Lambda function use stronger encryption keys generated by KMS like a7e63k4b-8loc-40db-a2a1-4d0en2cd8331, you can specify a KMS key ID.

  • disable_spill_encryption – (Optional) When set to True, disables spill encryption. Defaults to False so that data that is spilled to S3 is encrypted using AES-GCM – either using a randomly generated key or KMS to generate keys. Disabling spill encryption can improve performance, especially if your spill location uses server-side encryption.

  • default_ec2_image_owner – (Optional) When set, controls the default Amazon EC2 image owner that filters Amazon Machine Images (AMI). If you do not set this value and your query against the EC2 images table does not include a filter for owner, your results will include all public images.

Databases and tables

The Athena AWS CMDB connector makes the following databases and tables available for querying your AWS resource inventory. For more information on the columns available in each table, run a DESCRIBE database.table statement using the Athena console or API.

  • ec2 – This database contains Amazon EC2 related resources, including the following.

  • ebs_volumes – Contains details of your Amazon EBS volumes.

  • ec2_instances – Contains details of your EC2 Instances.

  • ec2_images – Contains details of your EC2 Instance images.

  • routing_tables – Contains details of your VPC Routing Tables.

  • security_groups – Contains details of your security groups.

  • subnets – Contains details of your VPC Subnets.

  • vpcs – Contains details of your VPCs.

  • emr – This database contains Amazon EMR related resources, including the following.

  • emr_clusters – Contains details of your EMR Clusters.

  • rds – This database contains Amazon RDS related resources, including the following.

  • rds_instances – Contains details of your RDS Instances.

  • s3 – This database contains RDS related resources, including the following.

  • buckets – Contains details of your Amazon S3 buckets.

  • objects – Contains details of your Amazon S3 objects, excluding their contents.

Required Permissions

For full details on the IAM policies that this connector requires, review the Policies section of the athena-aws-cmdb.yaml file. The following list summarizes the required permissions.

  • Amazon S3 write access – The connector requires write access to a location in Amazon S3 in order to spill results from large queries.

  • Athena GetQueryExecution – The connector uses this permission to fast-fail when the upstream Athena query has terminated.

  • S3 List – The connector uses this permission to list your Amazon S3 buckets and objects.

  • EC2 Describe – The connector uses this permission to describe resources such as your Amazon EC2 instances, security groups, VPCs, and Amazon EBS volumes.

  • EMR Describe / List – The connector uses this permission to describe your EMR clusters.

  • RDS Describe – The connector uses this permission to describe your RDS Instances.

Performance

Currently, the Athena AWS CMDB connector does not support parallel scans. Predicate pushdown is performed within the Lambda function. Where possible, partial predicates are pushed to the services being queried. For example, a query for the details of a specific Amazon EC2 instance calls the EC2 API with the specific instance ID to run a targeted describe operation.

License information

The Amazon Athena AWS CMDB connector project is licensed under the Apache-2.0 License.

Additional resources

For additional information about this connector, visit the corresponding site on GitHub.com.