The Data Catalog supports creating partition indexes to provide efficient lookup for specific partitions. For more information, see Creating partition indexes. The AWS Glue crawler creates partition indexes for Amazon S3 and Delta Lake targets by default.
-
Sign in to the AWS Management Console and open the AWS Glue console at https://console.aws.amazon.com/glue/
. -
Choose Crawlers under the Data Catalog.
-
When you define a crawler, the option to Create partition indexes automatically is enabled by default under Advanced options on the Set output and scheduling page.
To disable this option, you can unselect the checkbox Create partition indexes automatically in the console.
-
Complete the crawler configuration and choose Create crawler.
Usage notes for partition indexes
Tables created by the crawler do not have the variable
partition_filtering.enabledby default. For more information, see AWS Glue partition indexing and filtering.Creating partition indexes for encrypted partitions is not supported.
