Generating partition indexes
The Data Catalog supports creating partition indexes to provide efficient lookup for specific partitions. For more information, see Creating partition indexes. The AWS Glue crawler creates partition indexes for Amazon S3 and Delta Lake targets by deafult.
When you define a cralwer, the option to Create partition indexes automatically is enabled by default under Advanced options on the Set output and scheduling page.
To disable this option, you can unselect the checkbox Create partition indexes
automatically in the console. You can also disable this option by using the
crawler API, set the CreatePartitionIndex in the Configuration. The
default value is true.
Usage notes for partition indexes
Tables created by the crawler do not have the variable
partition_filtering.enabledby default. For more information, see AWS Glue partition indexing and filtering.Creating partition indexes for encrypted partitions is not supported.
