AWS Glue crawlers can be set up to run on a schedule or on demand. For more information, see Time-based schedules for jobs and crawlers in the AWS Glue Developer Guide.
If you have data that arrives for a partitioned table at a fixed time, you can set up
an AWS Glue crawler to run on schedule to detect and update table partitions. This can
eliminate the need to run a potentially long and expensive MSCK REPAIR
command or manually run an ALTER TABLE ADD PARTITION command. For more
information, see Table
partitions in the AWS Glue Developer Guide.
