Update an Amazon S3 Storage Lens dashboard - Amazon Simple Storage Service

Update an Amazon S3 Storage Lens dashboard

The Amazon S3 Storage Lens default dashboard is default-account-dashboard. This dashboard is preconfigured by Amazon S3 to help you visualize summarized insights and trends for your entire account's aggregated free and advanced metrics on the console. You can't modify the default dashboard's configuration scope, but you can upgrade the metrics selection from the free metrics to the paid advanced metrics and recommendations, configure the optional metrics export, or even disable the default dashboard. The default dashboard can't be deleted, and can only be disabled. For more information, see Using the S3 console.

Use the following steps to update an Amazon S3 Storage Lens dashboard on the Amazon S3 console.

Step 1: Update the dashboard scope

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.

  2. In the left navigation pane, choose Storage Lens, Dashboards.

  3. Choose the dashboard that you want to edit, and then choose Edit.

    The Edit dashboard page opens.

    Note

    You can't change the following:

    • The dashboard name

    • The home Region

    • The dashboard scope of the default dashboard, which is scoped to your entire account's storage

  4. (Optional) On the dashboard configuration page, in the General section, update and add tags to your dashboard.

    You can use tags to manage permissions for your dashboard and to track costs for S3 Storage Lens. For more information, see Controlling access using resource tags in the IAM User Guide and AWS-Generated Cost Allocation Tags in the AWS Billing User Guide.

    Note

    You can add up to 50 tags to your dashboard configuration.

  5. In the Dashboard scope section, do the following:

    1. Update the Regions and buckets that you want S3 Storage Lens to include or exclude in the dashboard.

      Note

      • You can either include or exclude Regions and buckets. This option is limited to Regions only when creating organization-level dashboards across member accounts in your organization.

      • You can choose up to 50 buckets to include or exclude.

    2. Update the buckets in your selected Regions that you want S3 Storage Lens to include or exclude. You can either include or exclude buckets, but not both. This option is not present when creating organization-level dashboards.

Step 2: Update the metrics selection

  1. In the Metrics selection section, choose the type of metrics that you want to aggregate for this dashboard.

    • To include free metrics aggregated at the bucket level and available for queries for 14 days, choose Free metrics.

    • To enable advanced metrics and other advanced options, choose Advanced metrics and recommendations. These options include advanced prefix aggregation, Amazon CloudWatch publishing, and contextual recommendations. Data is available for queries for 15 months. Advanced metrics and recommendations have an additional cost. For more information, see Amazon S3 pricing.

      For more information about advanced metrics and free metrics, see Metrics selection.

  2. Under Advanced metrics and recommendations features, select the options that you want to enable:

    • Advanced metrics

    • CloudWatch publishing

    • Prefix aggregation

    Important

    If you enable prefix aggregation for your S3 Storage Lens configuration, prefix-level metrics will not be published to CloudWatch. Only bucket, account, and organization-level S3 Storage Lens metrics are published to CloudWatch.

  3. If you enabled Advanced metrics, select the Advanced metrics categories that you want to display in your S3 Storage Lens dashboard:

    • Activity metrics

    • Detailed status code metrics

    • Advanced cost optimization metrics

    • Advanced data protection metrics

    For more information metrics categories, see Metrics categories. For a complete list of metrics, see Amazon S3 Storage Lens metrics glossary.

  4. If you chose to enable prefix aggregation, configure the following:

    1. Choose the minimum prefix threshold size for this dashboard.

      For example, a prefix threshold of 5 percent indicates that prefixes that make up 5 percent or more of the bucket's total storage size will be aggregated.

    2. Choose the prefix depth.

      This setting indicates the maximum number of levels up to which the prefixes are evaluated. The prefix depth must be less than 10.

    3. Enter a prefix delimiter character.

      This is the value used to identify each prefix level. The default value in Amazon S3 is the / character, but your storage structure might use other delimiter characters.

(Optional) Step 3: Export metrics for the dashboard

  1. In the Metrics export section, to create a metrics export that will be placed daily in a destination bucket of your choice, choose Enable. To disable the metrics export, choose Disable.

    The metrics export is in CSV or Apache Parquet format. It represents the same scope of data as your S3 Storage Lens dashboard data without the recommendations.

  2. If enabled, choose the output format of your daily metrics export: CSV or Apache Parquet.

    Parquet is an open source file format for Hadoop that stores nested data in a flat columnar format.

  3. Choose the destination S3 bucket for your metrics export.

    You can choose a bucket in the current account of the S3 Storage Lens dashboard. Or you can choose another AWS account if you have the destination bucket permissions and the destination bucket owner's account ID.

  4. Choose the destination S3 bucket (format: s3://bucket-name/prefix).

    The bucket must be in the home Region of your S3 Storage Lens dashboard. The S3 console shows you the Destination bucket permission that will be added by Amazon S3 to the destination bucket policy. Amazon S3 updates the bucket policy on the destination bucket to allow S3 to place data in that bucket.

  5. (Optional) To enable server-side encryption for your metrics export, choose Specify an encryption key. Then, choose the Encryption type: Amazon S3 managed keys (SSE-S3) or AWS Key Management Service key (SSE-KMS).

    You can choose between an Amazon S3 managed key (SSE-S3) and an AWS Key Management Service (AWS KMS) key (SSE-KMS).

  6. (Optional) To specify an AWS KMS key, you must choose a KMS key or enter a key Amazon Resource Name (ARN). Under AWS KMS key, specify your KMS key in one of the following ways:

    • To choose from a list of available KMS keys, choose Choose from your AWS KMS keys, and choose your KMS key from the list of available keys.

      Both the AWS managed key (aws/s3) and your customer managed keys appear in this list. For more information about customer managed keys, see Customer keys and AWS keys in the AWS Key Management Service Developer Guide.

      Note

      The AWS managed key (aws/S3) is not supported for SSE-KMS encryption with S3 Storage Lens.

    • To enter the KMS key ARN, choose Enter AWS KMS key ARN, and enter your KMS key ARN in the field that appears.

    • To create a new customer managed key in the AWS KMS console, choose Create a KMS key.

      If you choose a customer managed key, you must grant S3 Storage Lens permission to encrypt in the AWS KMS key policy. For more information, see Using an AWS KMS key to encrypt your metrics exports.

      For more information about creating an AWS KMS key, see Creating Keys in the AWS Key Management Service Developer Guide.

  7. Choose Save changes.

To gain further visibility into your storage, you can create one or more S3 Storage Lens groups and attach them to your dashboard. An S3 Storage Lens group is a custom defined filter for objects based on prefixes, suffixes, object tags, object size, object age, or a combination of these filters.

You can use S3 Storage Lens groups to gain granular visibility into large shared buckets, such as data lakes, to make better-informed business decisions. For example, you can streamline storage allocation and optimize cost reporting by breaking down storage usage to specific groups of objects for individual projects and cost centers within a bucket or across multiple buckets.

To use S3 Storage Lens groups, you must upgrade your dashboard to use advanced metrics and recommendations. For more information about S3 Storage Lens groups, see Working with S3 Storage Lens groups to filter and aggregate metrics.

The following example command updates a Amazon S3 Storage Lens dashboard configuration. To use these examples, replace the user input placeholders with your own information.

aws s3control put-storage-lens-configuration --account-id=111122223333 --config-id=example-dashboard-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json --tags=file://./tags.json
Example – Update a Amazon S3 Storage Lens configuration with advanced metrics and recommendations

The following examples shows you how to update the default S3 Storage Lens configuration with advanced metrics and recommendations in SDK for Java:

package aws.example.s3control;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3control.AWSS3Control;
import com.amazonaws.services.s3control.AWSS3ControlClient;
import com.amazonaws.services.s3control.model.AccountLevel;
import com.amazonaws.services.s3control.model.ActivityMetrics;
import com.amazonaws.services.s3control.model.BucketLevel;
import com.amazonaws.services.s3control.model.Format;
import com.amazonaws.services.s3control.model.Include;
import com.amazonaws.services.s3control.model.OutputSchemaVersion;
import com.amazonaws.services.s3control.model.PrefixLevel;
import com.amazonaws.services.s3control.model.PrefixLevelStorageMetrics;
import com.amazonaws.services.s3control.model.PutStorageLensConfigurationRequest;
import com.amazonaws.services.s3control.model.S3BucketDestination;
import com.amazonaws.services.s3control.model.SSES3;
import com.amazonaws.services.s3control.model.SelectionCriteria;
import com.amazonaws.services.s3control.model.StorageLensAwsOrg;
import com.amazonaws.services.s3control.model.StorageLensConfiguration;
import com.amazonaws.services.s3control.model.StorageLensDataExport;
import com.amazonaws.services.s3control.model.StorageLensDataExportEncryption;
import com.amazonaws.services.s3control.model.StorageLensTag;

import java.util.Arrays;
import java.util.List;

import static com.amazonaws.regions.Regions.US_WEST_2;

public class UpdateDefaultConfigWithPaidFeatures {

    public static void main(String[] args) {
        String configurationId = "default-account-dashboard"; // This configuration ID cannot be modified.
        String sourceAccountId = "111122223333";

        try {
            SelectionCriteria selectionCriteria = new SelectionCriteria()
                    .withDelimiter("/")
                    .withMaxDepth(5)
                    .withMinStorageBytesPercentage(10.0);
            PrefixLevelStorageMetrics prefixStorageMetrics = new PrefixLevelStorageMetrics()
                    .withIsEnabled(true)
                    .withSelectionCriteria(selectionCriteria);
            BucketLevel bucketLevel = new BucketLevel()
                    .withActivityMetrics(new ActivityMetrics().withIsEnabled(true))
                    .withPrefixLevel(new PrefixLevel().withStorageMetrics(prefixStorageMetrics));
            AccountLevel accountLevel = new AccountLevel()
                    .withActivityMetrics(new ActivityMetrics().withIsEnabled(true))
                    .withBucketLevel(bucketLevel);

            StorageLensConfiguration configuration = new StorageLensConfiguration()
                    .withId(configurationId)
                    .withAccountLevel(accountLevel)
                    .withIsEnabled(true);

            AWSS3Control s3ControlClient = AWSS3ControlClient.builder()
                    .withCredentials(new ProfileCredentialsProvider())
                    .withRegion(US_WEST_2)
                    .build();

            s3ControlClient.putStorageLensConfiguration(new PutStorageLensConfigurationRequest()
                    .withAccountId(sourceAccountId)
                    .withConfigId(configurationId)
                    .withStorageLensConfiguration(configuration)
            );

        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it and returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
    }
}
Note

Additional charges apply for advanced metrics and recommendations. For more information, see advanced metrics and recommendations.