SUS04-BP03 Use policies to manage the lifecycle of your datasets
Manage the lifecycle of all of your data and automatically enforce deletion to minimize the total storage required for your workload.
Common anti-patterns:
-
You manually delete data.
-
You do not delete any of your workload data.
-
You do not move data to more energy-efficient storage tiers based on its retention and access requirements.
Benefits of establishing this best practice: Using data lifecycle policies ensures efficient data access and retention in a workload.
Level of risk exposed if this best practice is not established: Medium
Implementation guidance
Datasets usually have different retention and access requirements during their lifecycle. For example, your application may need frequent access to some datasets for a limited period of time. After that, those datasets are infrequently accessed.
To efficiently manage your datasets throughout their lifecycle, configure lifecycle policies, which are rules that define how to handle datasets.
With Lifecycle configuration rules, you can tell the specific storage service to transition a dataset to more energy-efficient storage tiers, archive it, or delete it.
Implementation steps
-
Define handling procedures for each data class.
-
Set automated lifecycle policies to enforce lifecycle rules. Here are some examples of how to set up automated lifecycle policies for different AWS storage services:
Storage service How to set automated lifecycle policies You can use Amazon S3 Lifecycle to manage your objects throughout their lifecycle. If your access patterns are unknown, changing, or unpredictable, you can use Amazon S3 Intelligent-Tiering, which monitors access patterns and automatically moves objects that have not been accessed to lower-cost access tiers. You can leverage Amazon S3 Storage Lens metrics to identify optimization opportunities and gaps in lifecycle management.
You can use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of Amazon EBS snapshots and Amazon EBS-backed AMIs.
Amazon EFS lifecycle management automatically manages file storage for your file systems.
Amazon ECR lifecycle policies automate the cleanup of your container images by expiring images based on age or count.
You can use an object lifecycle policy that governs how long objects should be stored in the MediaStore container.
-
Delete unused volumes, snapshots, and data that is out of its retention period. Leverage native service features like Amazon DynamoDB Time To Live or Amazon CloudWatch log retention for deletion.
-
Aggregate and compress data where applicable based on lifecycle rules.
Resources
Related documents:
Related videos:
