# Optimizing costs of Amazon Keyspaces tables This section covers best practices on how to optimize costs for your existing Amazon Keyspaces tables. You should look at the following strategies to see which cost optimization strategy best suits your needs and approach them iteratively. Each strategy provides an overview of what might be impacting your costs, how to look for opportunities to optimize costs, and prescriptive guidance on how to implement these best practices to help you save. **Topics** + [Evaluate your costs at the table level](CostOptimization_TableLevelCostAnalysis.md) + [Evaluate your table's capacity mode](CostOptimization_TableCapacityMode.md) + [Evaluate your table's Application Auto Scaling settings](CostOptimization_AutoScalingSettings.md) + [Identify your unused resources to optimize costs in Amazon Keyspaces](CostOptimization_UnusedResources.md) + [Evaluate your table usage patterns to optimize performance and cost](CostOptimization_TableUsagePatterns.md) + [Evaluate your provisioned capacity for right-sized provisioning](CostOptimization_RightSizedProvisioning.md) # Evaluate your costs at the table level The Cost Explorer tool found within the AWS Management Console allows you to see costs broken down by type, for example read, write, storage, and backup charges. You can also see these costs summarized by period such as month or day. One common challenge with Cost Explorer is that you can't review the costs of only one particular table easily, because Cost Explorer doesn't let you filter or group by costs of a specific table. You can view the metric **Billable table size (Bytes)** of each table in the Amazon Keyspaces console on the table's **Monitor** tab. If you need more cost related information per table, this section shows you how to use [tagging](tagging-keyspaces.md) to perform individual table cost analysis in Cost Explorer. **Topics** + [How to view the costs of a single Amazon Keyspaces table](#CostOptimization_TableLevelCostAnalysis_ViewInfo) + [Cost Explorer's default view](#CostOptimization_TableLevelCostAnalysis_CostExplorer) + [How to use and apply table tags in Cost Explorer](#CostOptimization_TableLevelCostAnalysis_Tagging) ## How to view the costs of a single Amazon Keyspaces table You can see basic information about an Amazon Keyspaces table in the console, including the primary key schema, the billable table size, and capacity related metrics. You can use the size of the table to calculate the monthly storage cost for the table. For example, \$10.25 per GB in the US East (N. Virginia) AWS Region. If the table is using provisioned capacity mode, the current read capacity unit (RCU) and write capacity unit (WCU) settings are returned as well. You can use this information to calculate the current read and write costs for the table. Note that these costs could change, especially if you have configured the table with Amazon Keyspaces automatic scaling. ## Cost Explorer's default view The default view in Cost Explorer provides charts showing the cost of consumed resources, for example throughput and storage. You can choose to group these costs by period, such as totals by month or by day. The costs of storage, reads, writes, and other categories can be broken out and compared as well. ![\[Image showing the cost of consumed resources in the Cost Explorer view.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/CostExplorerView.png) ## How to use and apply table tags in Cost Explorer By default, Cost Explorer does not provide a summary of the costs for any one specific table, because it combines the costs of multiple tables into a total. However, you can use [AWS resource tagging](https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html) to identify each table by a metadata tag. Tags are key-value pairs that you can use for a variety of purposes, for example to identify all resources belonging to a project or department. For more information, see [Working with tags and labels for Amazon Keyspaces resources](tagging-keyspaces.md). For this example, we use a table with the name **MyTable**. 1. Set a tag with the key of **table\$1name** and the value of **MyTable**. 1. [Activate the tag within Cost Explorer](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html) and then filter on the tag value to gain more visibility into each table's costs. **Note** It may take one or two days for the tag to start appearing in Cost Explorer You can set metadata tags yourself in the console, or programmatically with CQL, the AWS CLI, or the AWS SDK. Consider requiring a **table\$1name** tag to be set as part of your organization’s new table creation process. For more information, see [Create cost allocation reports using tags for Amazon Keyspaces](CostAllocationReports.md). # Evaluate your table's capacity mode This section provides an overview of how to select the appropriate capacity mode for your Amazon Keyspaces table. Each mode is tuned to meet the needs of a different workload in terms of responsiveness to change in throughput, as well as how that usage is billed. You must balance these factors when making your decision. **Topics** + [What table capacity modes are available](#CostOptimization_TableCapacityMode_Overview) + [When to select on-demand capacity mode](#CostOptimization_TableCapacityMode_OnDemand) + [When to select provisioned capacity mode](#CostOptimization_TableCapacityMode_Provisioned) + [Additional factors to consider when choosing a table capacity mode](#CostOptimization_TableCapacityMode_AdditionalFactors) ## What table capacity modes are available When you create an Amazon Keyspaces table, you must select either on-demand or provisioned capacity mode. For more information, see [Configure read/write capacity modes in Amazon Keyspaces](ReadWriteCapacityMode.md). **On-demand capacity mode** The on-demand capacity mode is designed to eliminate the need to plan or provision the capacity of your Amazon Keyspaces table. In this mode, your table instantly accommodates requests without the need to scale any resources up or down (up to twice the previous peak throughput of the table). On-demand tables are billed by counting the number of actual requests against the table, so you only pay for what you use rather than what has been provisioned. **Provisioned capacity mode** The provisioned capacity mode is a more traditional model where you can define how much capacity the table has available for requests either directly or with the assistance of Application Auto Scaling. Because a specific capacity is provisioned for the table at any given time, billing is based off of the capacity provisioned rather than the number of requests. Going over the allocated capacity can also cause the table to reject requests and reduce the experience of your application's users. Provisioned capacity mode requires a balance between not over-provisioning or under provisioning the table to achieve both, low occurrence of insufficient throughput capacity errors, and optimized costs. ## When to select on-demand capacity mode When optimizing for cost, on-demand mode is your best choice when you have an unpredictable workload similar to the one shown in the following graph. These factors contribute to this type of workload: + Unpredictable request timing (resulting in traffic spikes) + Variable volume of requests (resulting from batch workloads) + Drops to zero or below 18% of the peak for a given hour (resulting from development or test environments) ![\[Image showing a spiky workload with random peaks in traffic.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/TableCapacityModeOnDemand.png) For workloads with the above characteristics, using Application Auto Scaling to maintain enough capacity for the table to respond to spikes in traffic may lead to undesirable outcomes. Either the table could be over-provisioned and costing more than necessary, or the table could be under provisioned and requests are leading to unnecessary low capacity throughput errors. In cases like this, on-demand tables are the better choice. Because on-demand tables are billed by request, there is nothing further you need to do at the table level to optimize for cost. You should regularly evaluate your on-demand tables to verify the workload still has the above characteristics. If the workload has stabilized, consider changing to provisioned mode to maintain cost optimization. ## When to select provisioned capacity mode An ideal workload for provisioned capacity mode is one with a more predictable usage pattern like shown in the graph below. The following factors contribute to a predictable workload: + Predicable/cyclical traffic for a given hour or day + Limited short term bursts of traffic ![\[Image showing a fairly predictable workload with limited peaks in traffic.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/TableCapacityModeProvisioned.png) Since the traffic volumes within a given time or day are more stable, you can set the provisioned capacity relatively close to the actual consumed capacity of the table. Cost optimizing a provisioned capacity table is ultimately an exercise in getting the provisioned capacity (blue line) as close to the consumed capacity (orange line) as possible without increasing `ThrottledRequests` events for the table. The space between the two lines is both, wasted capacity as well as insurance against a bad user experience due to insufficient throughput capacity errors. Amazon Keyspaces provides Application Auto Scaling for provisioned capacity tables, which automatically balances this on your behalf. You can track your consumed capacity throughout the day and configure the provisioned capacity of the table based on a handful of variables. **Minimum capacity units** You can set the minimum capacity of a table to limit the occurrence of insufficient throughput capacity errors, but it doesn't reduce the cost of the table. If your table has periods of low usage followed by a sudden burst of high usage, setting the minimum can prevent Application Auto Scaling from setting the table capacity too low. **Maximum capacity units** You can set the maximum capacity of a table to limit a table scaling higher than intended. Consider applying a maximum for development or test tables, where large-scale load testing is not desired. You can set a maximum for any table, but be sure to regularly evaluate this setting against the table baseline when using it in production, to prevent accidental insufficient throughput capacity errors. **Target utilization** Setting the target utilization of the table is the primary means of cost optimization for a provisioned capacity table. Setting a lower percent value here increases how much the table is over-provisioned, increasing cost, but reducing the risk of insufficient throughput capacity errors. Setting a higher percentage value decreases by how much the table is over-provisioned, but increases the risk of insufficient throughput capacity errors. ## Additional factors to consider when choosing a table capacity mode When deciding between the two capacity modes, there are some additional factors worth considering. When deciding between the two table modes, consider how much this additional discount affects the cost of the table. In many cases, even a relatively unpredictable workload can be more cost effective to run on an over-provisioned provisioned capacity table with reserved capacity. **Improving predictability of your workload** In some situations, a workload may seemingly have both, a predictable and an unpredictable pattern. While this can be easily supported with an on-demand table, costs will likely be lower if the unpredictable patterns in the workload can be improved. One of the most common causes of these patterns are batch imports. This type of traffic can often exceed the baseline capacity of the table to such a degree that insufficient throughput capacity errors would occur if it were to run. To keep a workload like this running on a provisioned capacity table, consider the following options: + If the batch occurs at scheduled times, you can schedule an increase to your application auto- scaling capacity before it runs. + If the batch occurs randomly, consider trying to extend the time it takes to run rather than executing as fast as possible. + Add a ramp up period to the import, where the velocity of the import starts small but is slowly increased over a few minutes until Application Auto Scaling has had the opportunity to start adjusting table capacity. # Evaluate your table's Application Auto Scaling settings This section provides an overview of how to evaluate the Application Auto Scaling settings of your Amazon Keyspaces tables. [Amazon Keyspaces Application Auto Scaling](autoscaling.md) is a feature that manages table throughput based on your application traffic and your target utilization metric. This ensures your tables have the required capacity required for your application patterns. The Application Auto Scaling service monitors your current table utilization and compares it to the target utilization value: `TargetValue`. It notifies you if it is time to increase or decrease the allocated capacity. **Topics** + [Understanding your Application Auto Scaling settings](#CostOptimization_AutoScalingSettings_UnderProvisionedTables) + [How to identify tables with low target utilization (<=50%)](#CostOptimization_AutoScalingSettings_IdentifyLowUtilization) + [How to address workloads with seasonal variance](#CostOptimization_AutoScalingSettings_SeasonalVariance) + [How to address spiky workloads with unknown patterns](#CostOptimization_AutoScalingSettings_UnknownPatterns) + [How to address workloads with linked applications](#CostOptimization_AutoScalingSettings_BetweenRanges) ## Understanding your Application Auto Scaling settings Defining the correct value for the target utilization, initial step, and final values is an activity that requires involvement from your operations team. This allows you to properly define the values based on historical application usage, which is used to trigger the Application Auto Scaling policies. The utilization target is the percentage of your total capacity that needs to be met during a period of time before the Application Auto Scaling rules apply. When you set a **high utilization target (a target around 90%)** it means your traffic needs to be higher than 90% for a period of time before the Application Auto Scaling is activated. You should not use a high utilization target unless your application is very constant and doesn’t receive spikes in traffic. When you set a very **low utilization (a target less than 50%)** it means your application would need to reach 50% of the provisioned capacity before it triggers an Application Auto Scaling policy. Unless your application traffic grows at a very aggressive rate, this usually translates into unused capacity and wasted resources. ## How to identify tables with low target utilization (<=50%) You can use either the AWS CLI or AWS Management Console to monitor and identify the `TargetValues` for your Application Auto Scaling policies in your Amazon Keyspaces resources. **Note** When you're using multi-Region tables in provisioned capacity mode with Amazon Keyspaces auto scaling, make sure to use the Amazon Keyspaces API operations to configure auto scaling. The underlying Application Auto Scaling API operations that Amazon Keyspaces calls on your behalf don't have multi-Region capabilities. For more information, see [View the provisioned capacity and auto scaling settings for a multi-Region table in Amazon Keyspaces](tables-mrr-view.md). ------ #### [ AWS CLI ] 1. Return the entire list of resources by running the following command: ``` aws application-autoscaling describe-scaling-policies --service-namespace cassandra ``` This command will return the entire list of Application Auto Scaling policies that are issued to any Amazon Keyspaces resource. If you only want to retrieve the resources from a particular table, you can add the `–resource-id parameter`. For example: ``` aws application-autoscaling describe-scaling-policies --service-namespace cassandra --resource-id "keyspace/keyspace-name/table/table-name” ``` 1. Return only the auto scaling policies for a particular table by running the following command ``` aws application-autoscaling describe-scaling-policies --service-namespace cassandra --resource-id "keyspace/keyspace-name/table/table-name” ``` The values for the Application Auto Scaling policies are highlighted below. You need to ensure that the target value is greater than 50% to avoid over-provisioning. You should obtain a result similar to the following: ``` { "ScalingPolicies": [ { "PolicyARN": "arn:aws:autoscaling:us-east-1:111122223333:scalingPolicy::resource/keyspaces/table/table-name-scaling-policy", "PolicyName": $”, "ServiceNamespace": "cassandra", "ResourceId": "keyspace/keyspace-name/table/table-name", "ScalableDimension": "cassandra:table:WriteCapacityUnits", "PolicyType": "TargetTrackingScaling", "TargetTrackingScalingPolicyConfiguration": { "TargetValue": 70.0, "PredefinedMetricSpecification": { "PredefinedMetricType": "KeyspacesWriteCapacityUtilization" } }, "Alarms": [ ... ], "CreationTime": "2022-03-04T16:23:48.641000+10:00" }, { "PolicyARN": "arn:aws:autoscaling:us-east-1:111122223333:scalingPolicy::resource/keyspaces/table/table-name-scaling-policy", "PolicyName":$”, "ServiceNamespace": "cassandra", "ResourceId": "keyspace/keyspace-name/table/table-name", "ScalableDimension": "cassandra:table:ReadCapacityUnits", "PolicyType": "TargetTrackingScaling", "TargetTrackingScalingPolicyConfiguration": { "TargetValue": 70.0, "PredefinedMetricSpecification": { "PredefinedMetricType": "KeyspacesReadCapacityUtilization" } }, "Alarms": [ ... ], "CreationTime": "2022-03-04T16:23:47.820000+10:00" } ] } ``` ------ #### [ AWS Management Console ] 1. Log into the AWS Management Console and navigate to the CloudWatch service page at [Getting Started with the AWS Management Console](https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/getting-started.html). Select the appropriate AWS Region if necessary. 1. On the left navigation bar, select **Tables**. On the **Tables** page, select the table's **Name**. 1. On the **Table Details** page on the **Capacity** tab, review your table's Application Auto Scaling settings. ------ If your target utilization values are less than or equal to 50%, you should explore your table utilization metrics to see if they are [under-provisioned or over-provisioned](CostOptimization_RightSizedProvisioning.md). ## How to address workloads with seasonal variance Consider the following scenario: your application is operating under a minimum average value most of the time, but the utilization target is low so your application can react quickly to events that happen at certain hours in the day and you have enough capacity and avoid getting throttled. This scenario is common when you have an application that is very busy during normal office hours (9 AM to 5 PM) but then it works at a base level during after hours. Since some users start to connect before 9 am, the application uses this low threshold to ramp up quickly to get to the *required* capacity during peak hours. This scenario could look like this: + Between 5 PM and 9 AM the `ConsumedWriteCapacityUnits` units stay between 90 and 100 + Users start to connect to the application before 9 AM and the capacity units increases considerably (the maximum value you’ve seen is 1500 WCU) + On average, your application usage varies between 800 to 1200 during working hours If the previous scenario applies to your application, consider using [scheduled application auto scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/examples-scheduled-actions.html), where your table could still have an Application Auto Scaling rule configured, but with a less aggressive target utilization that only provisions the extra capacity at the specific intervals you require. You can use the AWS CLI to execute the following steps to create a scheduled auto scaling rule that executes based on the time of day and the day of the week. 1. Register your Amazon Keyspaces table as a scalable target with Application Auto Scaling. A scalable target is a resource that Application Auto Scaling can scale out or in. ``` aws application-autoscaling register-scalable-target \ --service-namespace cassandra \ --scalable-dimension cassandra:table:WriteCapacityUnits \ --resource-id keyspace/keyspace-name/table/table-name \ --min-capacity 90 \ --max-capacity 1500 ``` 1. Set up scheduled actions according to your requirements. You need two rules to cover the scenario: one to scale up and another to scale down. The first rule to scale up the scheduled action is shown in the following example. ``` aws application-autoscaling put-scheduled-action \ --service-namespace cassandra \ --scalable-dimension cassandra:table:WriteCapacityUnits \ --resource-id keyspace/keyspace-name/table/table-name \ --scheduled-action-name my-8-5-scheduled-action \ --scalable-target-action MinCapacity=800,MaxCapacity=1500 \ --schedule "cron(45 8 ? * MON-FRI *)" \ --timezone "Australia/Brisbane" ``` The second rule to scale down the scheduled action is shown in this example. ``` aws application-autoscaling put-scheduled-action \ --service-namespace cassandra \ --scalable-dimension cassandra:table:WriteCapacityUnits \ --resource-id keyspace/keyspace-name/table/table-name \ --scheduled-action-name my-5-8-scheduled-down-action \ --scalable-target-action MinCapacity=90,MaxCapacity=1500 \ --schedule "cron(15 17 ? * MON-FRI *)" \ --timezone "Australia/Brisbane" ``` 1. Run the following command to validate both rules have been activated: ``` aws application-autoscaling describe-scheduled-actions --service-namespace cassandra ``` You should get a result like this: ``` { "ScheduledActions": [ { "ScheduledActionName": "my-5-8-scheduled-down-action", "ScheduledActionARN": "arn:aws:autoscaling:us-east-1:111122223333:scheduledAction::resource/keyspaces/table/table-name:scheduledActionName/my-5-8-scheduled-down-action", "ServiceNamespace": "cassandra", "Schedule": "cron(15 17 ? * MON-FRI *)", "Timezone": "Australia/Brisbane", "ResourceId": "keyspace/keyspace-name/table/table-name", "ScalableDimension": "cassandra:table:WriteCapacityUnits", "ScalableTargetAction": { "MinCapacity": 90, "MaxCapacity": 1500 }, "CreationTime": "2022-03-15T17:30:25.100000+10:00" }, { "ScheduledActionName": "my-8-5-scheduled-action", "ScheduledActionARN": "arn:aws:autoscaling:us-east-1:111122223333:scheduledAction::resource/keyspaces/table/table-name:scheduledActionName/my-8-5-scheduled-action", "ServiceNamespace": "cassandra", "Schedule": "cron(45 8 ? * MON-FRI *)", "Timezone": "Australia/Brisbane", "ResourceId": "keyspace/keyspace-name/table/table-name", "ScalableDimension": "cassandra:table:WriteCapacityUnits", "ScalableTargetAction": { "MinCapacity": 800, "MaxCapacity": 1500 }, "CreationTime": "2022-03-15T17:28:57.816000+10:00" } ] } ``` The following picture shows a sample workload that always keeps the 70% target utilization. Notice how the auto scaling rules are still applying and the throughput is not getting reduced. ![\[A graph that shows write usage in units per second comparing provisioned to consumed capacity over the period of one day.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/AutoScalingSettings3.png) Zooming in, we can see there was a spike in the application that triggered the 70% auto scaling threshold, forcing the autoscaling to kick in and provide the extra capacity required for the table. The scheduled auto scaling action will affect maximum and minimum values, and it's your responsibility to set them up. ![\[A more detailed view of the graph that shows write usage in units per second comparing provisioned to consumed capacity, zooming in on a specific time.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/AutoScalingSettings4.png) ![\[Showing the detailed view of the graph that shows write usage in units per second comparing provisioned to consumed capacity over the period of one day.\]](http://docs.aws.amazon.com/keyspaces/latest/devguide/images/CostOptimization/AutoScalingSettings5.png) ## How to address spiky workloads with unknown patterns In this scenario, the application uses a very low utilization target, because you don’t know the application patterns yet, and you want to ensure your workload is not experiencing low capacity throughput errors. Consider using [on-demand capacity mode](ReadWriteCapacityMode.OnDemand.md) instead. On-demand tables are perfect for spiky workloads where you don’t know the traffic patterns. With on-demand capacity mode, you pay per request for the data reads and writes your application performs on your tables. You do not need to specify how much read and write throughput you expect your application to perform, as Amazon Keyspaces instantly accommodates your workloads as they ramp up or down. ## How to address workloads with linked applications In this scenario, the application depends on other systems, like batch processing scenarios where you can have big spikes in traffic according to events in the application logic. Consider developing custom application auto-scaling logic that reacts to those events where you can increase table capacity and `TargetValues` depending on your specific needs. You could benefit from Amazon EventBridge and use a combination of AWS services like Λ and Step Functions to react to your specific application needs. # Identify your unused resources to optimize costs in Amazon Keyspaces This section provides an overview of how to evaluate your unused resources regularly. As your application requirements evolve, you should ensure no resources are unused and contributing to unnecessary Amazon Keyspaces costs. The procedures described below use Amazon CloudWatch metrics to identify unused resources and take action to reduce costs. You can monitor Amazon Keyspaces using CloudWatch, which collects and processes raw data from Amazon Keyspaces into readable, near real-time metrics. These statistics are retained for a period of time, so that you can access historical information to better understand your utilization. By default, Amazon Keyspaces metric data is sent to CloudWatch automatically. For more information, see [What is Amazon CloudWatch?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) and [Metrics retention](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#metrics-retention) in the *Amazon CloudWatch User Guide*. **Topics** + [How to identify unused resources](#CostOptimization_UnusedResources_Identifying) + [Identifying unused table resources](#CostOptimization_UnusedResources_Tables) + [Cleaning up unused table resources](#CostOptimization_UnusedResources_Tables_Cleanup) + [Cleaning up unused point-in-time recovery (PITR) backups](#CostOptimization_UnusedResources_Backups) ## How to identify unused resources To identify unused tables you can take a look at the following CloudWatch metrics over a period of 30 days to understand if there are any active reads or writes on a specific table: **`ConsumedReadCapacityUnits`** The number of read capacity units consumed over the specified time period, so you can track how much consumed capacity you have used. You can retrieve the total consumed read capacity for a table. **`ConsumedWriteCapacityUnits`** The number of write capacity units consumed over the specified time period, so you can track how much consumed capacity you have used. You can retrieve the total consumed write capacity for a table. ## Identifying unused table resources Amazon CloudWatch is a monitoring and observability service which provides the Amazon Keyspaces table metrics you can use to identify unused resources. CloudWatch metrics can be viewed through the AWS Management Console as well as through the AWS Command Line Interface. ------ #### [ AWS Command Line Interface ] To view your tables metrics through the AWS Command Line Interface, you can use the following commands. 1. First, evaluate your table's reads: **Note** If the table name is not unique within your account, you must also specify the name of the keyspace. ``` aws cloudwatch get-metric-statistics --metric-name ConsumedReadCapacityUnits --start-time --end-time --period --namespace AWS/Cassandra --statistics Sum -- dimensions Name=TableName,Value= ``` To avoid falsely identifying a table as unused, evaluate metrics over a longer period. Choose an appropriate start-time and end-time range, such as ** 30 days**, and an appropriate period, such as **86400**. In the returned data, any **Sum** above **0** indicates that the table you are evaluating received read traffic during that period. The following result shows a table receiving read traffic in the evaluated period: ``` { "Timestamp": "2022-08-25T19:40:00Z", "Sum": 36023355.0, "Unit": "Count" }, { "Timestamp": "2022-08-12T19:40:00Z", "Sum": 38025777.5, "Unit": "Count" }, ``` The following result shows a table not receiving read traffic in the evaluated period: ``` { "Timestamp": "2022-08-01T19:50:00Z", "Sum": 0.0, "Unit": "Count" }, { "Timestamp": "2022-08-20T19:50:00Z", "Sum": 0.0, "Unit": "Count" }, ``` 1. Next, evaluate your table’s writes: ``` aws cloudwatch get-metric-statistics --metric-name ConsumedWriteCapacityUnits --start-time --end-time --period --namespace AWS/Cassandra --statistics Sum -- dimensions Name=TableName,Value= ``` To avoid falsely identifying a table as unused, you will want to evaluate metrics over a longer period. Choose an appropriate start-time and end-time range, such as **30 days**, and an appropriate period, such as **86400**. In the returned data, any **Sum** above **0** indicates that the table you are evaluating received read traffic during that period. The following result shows a table receiving write traffic in the evaluated period: ``` { "Timestamp": "2022-08-19T20:15:00Z", "Sum": 41014457.0, "Unit": "Count" }, { "Timestamp": "2022-08-18T20:15:00Z", "Sum": 40048531.0, "Unit": "Count" }, ``` The following result shows a table not receiving write traffic in the evaluated period: ``` { "Timestamp": "2022-07-31T20:15:00Z", "Sum": 0.0, "Unit": "Count" }, { "Timestamp": "2022-08-19T20:15:00Z", "Sum": 0.0, "Unit": "Count" }, ``` ------ #### [ AWS Management Console ] The following steps allow you to evaluate your resource utilization through the AWS Management Console. 1. Log into the AWS Management Console and navigate to the CloudWatch service page at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). Select the appropriate AWS Region in the top right of the console, if necessary. 1. On the left navigation bar, locate the **Metrics** section and choose **All metrics**. 1. The action above opens a dashboard with two panels. In the top panel, you can see currently graphed metrics. On the bottom you can select the metrics available to graph. Choose Amazon Keyspaces in the bottom panel. 1. In the Amazon Keyspaces metrics selection panel, choose the **Table Metrics** category to show the metrics for your tables in the current region. 1. Identify your table name by scrolling down the menu, then choose the metrics `ConsumedReadCapacityUnits` and `ConsumedWriteCapacityUnits` for your table. 1. Choose the **Graphed metrics (2)** tab and adjust the **Statistic** column to **Sum**. 1. To avoid falsely identifying a table as unused, evaluate the table metrics over a longer period. At the top of the graph panel, choose an appropriate time frame, such as 1 month, to evaluate your table. Choose **Custom**, choose **1 Months** in the drop-down menu, and choose **Apply**. 1. Evaluate the graphed metrics for your table to determine if it is being used. Metrics that have gone above **0** indicate that a table has been used during the evaluated time period. A flat graph at **0** for both read and write indicates that a table is unused. ------ ## Cleaning up unused table resources If you have identified unused table resources, you can reduce their ongoing costs in the following ways. **Note** If you have identified an unused table but would still like to keep it available in case it needs to be accessed in the future, consider switching it to on-demand mode. Otherwise, you can consider deleting the table. **Capacity modes** Amazon Keyspaces charges for reading, writing, and storing data in your Amazon Keyspaces tables. Amazon Keyspaces has [two capacity modes](ReadWriteCapacityMode.md), which come with specific billing options for processing reads and writes on your tables: on-demand and provisioned. The read/write capacity mode controls how you are charged for read and write throughput and how you manage capacity. For on-demand mode tables, you don't need to specify how much read and write throughput you expect your application to perform. Amazon Keyspaces charges you for the reads and writes that your application performs on your tables in terms of read request units and write request units. If there is no activity on your table, you do not pay for throughput but you still incur a storage charge. **Deleting tables** If you’ve discovered an unused table and would like to delete it, consider to make a backup or export the data first. Backups taken through AWS Backup can leverage cold storage tiering, further reducing cost. Refer to the [Managing backup plans](https://docs.aws.amazon.com/aws-backup/latest/devguide/about-backup-plans) documentation for information on how to use a lifecycle to move your backup to cold storage. After your table has been backed up, you may choose to delete it either through the AWS Management Console or through the AWS Command Line Interface. ## Cleaning up unused point-in-time recovery (PITR) backups Amazon Keyspaces offers Point-in-time recovery, which provides continuous backups for 35 days to help you protect against accidental writes or deletes. PITR backups have costs associated with them. Refer to the documentation at [Backup and restore data with point-in-time recovery for Amazon Keyspaces](PointInTimeRecovery.md) to determine if your tables have backups enabled that may no longer be needed. # Evaluate your table usage patterns to optimize performance and cost This section provides an overview of how to evaluate if you are efficiently using your Amazon Keyspaces tables. There are certain usage patterns which are not optimal for Amazon Keyspaces, and they allow room for optimization from both a performance and a cost perspective. **Topics** + [Perform fewer strongly-consistent read operations](#CostOptimization_TableUsagePatterns_StronglyConsistentReads) + [Enable Time to Live (TTL)](#CostOptimization_TableUsagePatterns_TTL) ## Perform fewer strongly-consistent read operations Amazon Keyspaces allows you to configure [read consistency](consistency.md#ReadConsistency) on a per-request basis. Read requests are eventually consistent by default. Eventually consistent reads are charged at 0.5 RCU for up to 4 KB of data. Most parts of distributed workloads are flexible and can tolerate eventual consistency. However, there can be access patterns requiring strongly consistent reads. Strongly consistent reads are charged at 1 RCU for up to 4 KB of data, essentially doubling your read costs. Amazon Keyspaces provides you with the flexibility to use both consistency models on the same table. You can evaluate your workload and application code to confirm if strongly consistent reads are used only where required. ## Enable Time to Live (TTL) [Time to Live (TTL)](TTL.md) helps you simplify your application logic and optimize the price of storage by expiring data from tables automatically. Data that you no longer need is automatically deleted from your table based on the Time to Live value that you set. # Evaluate your provisioned capacity for right-sized provisioning This section provides an overview of how to evaluate if you have right-sized provisioning on your Amazon Keyspaces tables. As your workload evolves, you should modify your operational procedures appropriately, especially when your Amazon Keyspaces table is configured in provisioned mode and you have the risk to over-provision or under-provision your tables. The procedures described in this section require statistical information that should be captured from the Amazon Keyspaces tables that are supporting your production application. To understand your application behavior, you should define a period of time that is significant enough to capture the data seasonality of your application. For example, if your application shows weekly patterns, using a three week period should give you enough room for analysing application throughput needs. If you don’t know where to start, use at least one month’s worth of data usage for the calculations below. While evaluating capacity, for Amazon Keyspaces tables you can configure **Read Capacity Units (RCUs)** and **Write Capacity Units (WCU)** independently. **Topics** + [How to retrieve consumption metrics from your Amazon Keyspaces tables](#CostOptimization_RightSizedProvisioning_ConsumptionMetrics) + [How to identify under-provisioned Amazon Keyspaces tables](#CostOptimization_RightSizedProvisioning_UnderProvisionedTables) + [How to identify over-provisioned Amazon Keyspaces tables](#CostOptimization_RightSizedProvisioning_OverProvisionedTables) ## How to retrieve consumption metrics from your Amazon Keyspaces tables To evaluate the table capacity, monitor the following CloudWatch metrics and select the appropriate dimension to retrieve table information: | Read Capacity Units | Write Capacity Units | | --- | --- | | `ConsumedReadCapacityUnits` | `ConsumedWriteCapacityUnits` | | `ProvisionedReadCapacityUnits` | `ProvisionedWriteCapacityUnits` | | `ReadThrottleEvents` | `WriteThrottleEvents` | You can do this either through the AWS CLI or the AWS Management Console. ------ #### [ AWS CLI ] Before you retrieve the table consumption metrics, you need to start by capturing some historical data points using the CloudWatch API. Start by creating two files: `write-calc.json` and `read-calc.json`. These files represent the calculations for the table. You need to update some of the fields, as indicated in the table below, to match your environment. **Note** If the table name is not unique within your account, you must also specify the name of the keyspace. | Field Name | Definition | Example | | --- | --- | --- | | | The name of the table that you are analysing | SampleTable | | | The period of time that you are using to evaluate the utilization target, based in seconds | For a 1-hour period you should specify: 3600 | | | The beginning of your evaluation interval, specified in ISO8601 format | 2022-02-21T23:00:00 | | | The end of your evaluation interval, specified in ISO8601 format | 2022-02-22T06:00:00 | The write calculations file retrieves the number of WCU provisioned and consumed in the time period for the date range specified. It also generates a utilization percentage that can be used for analysis. The full content of the `write-calc.json` file should look like in the following example. ``` { "MetricDataQueries": [ { "Id": "provisionedWCU", "MetricStat": { "Metric": { "Namespace": "AWS/Cassandra", "MetricName": "ProvisionedWriteCapacityUnits", "Dimensions": [ { "Name": "TableName", "Value": "" } ] }, "Period": , "Stat": "Average" }, "Label": "Provisioned", "ReturnData": false }, { "Id": "consumedWCU", "MetricStat": { "Metric": { "Namespace": "AWS/Cassandra", "MetricName": "ConsumedWriteCapacityUnits", "Dimensions": [ { "Name": "TableName", "Value": """ } ] }, "Period": , "Stat": "Sum" }, "Label": "", "ReturnData": false }, { "Id": "m1", "Expression": "consumedWCU/PERIOD(consumedWCU)", "Label": "Consumed WCUs", "ReturnData": false }, { "Id": "utilizationPercentage", "Expression": "100*(m1/provisionedWCU)", "Label": "Utilization Percentage", "ReturnData": true } ], "StartTime": "", "EndTime": "", "ScanBy": "TimestampDescending", "MaxDatapoints": 24 } ``` The read calculations file uses a similar metrics. This file retrieves how many RCUs were provisioned and consumed during the time period for the date range specified. The contents of the `read-calc.json` file should look like in this example. ``` { "MetricDataQueries": [ { "Id": "provisionedRCU", "MetricStat": { "Metric": { "Namespace": "AWS/Cassandra", "MetricName": "ProvisionedReadCapacityUnits", "Dimensions": [ { "Name": "TableName", "Value": "" } ] }, "Period": , "Stat": "Average" }, "Label": "Provisioned", "ReturnData": false }, { "Id": "consumedRCU", "MetricStat": { "Metric": { "Namespace": "AWS/Cassandra", "MetricName": "ConsumedReadCapacityUnits", "Dimensions": [ { "Name": "TableName", "Value": "" } ] }, "Period": , "Stat": "Sum" }, "Label": "", "ReturnData": false }, { "Id": "m1", "Expression": "consumedRCU/PERIOD(consumedRCU)", "Label": "Consumed RCUs", "ReturnData": false }, { "Id": "utilizationPercentage", "Expression": "100*(m1/provisionedRCU)", "Label": "Utilization Percentage", "ReturnData": true } ], "StartTime": "", "EndTime": "", "ScanBy": "TimestampDescending", "MaxDatapoints": 24 } ``` Once you've created the files, you can start retrieving utilization data. 1. To retrieve the write utilization data, issue the following command: ``` aws cloudwatch get-metric-data --cli-input-json file://write-calc.json ``` 1. To retrieve the read utilization data, issue the following command: ``` aws cloudwatch get-metric-data --cli-input-json file://read-calc.json ``` The result for both queries is a series of data points in JSON format that can be used for analysis. Your results depend on the number of data points you specified, the period, and your own specific workload data. It could look like in the following example. ``` { "MetricDataResults": [ { "Id": "utilizationPercentage", "Label": "Utilization Percentage", "Timestamps": [ "2022-02-22T05:00:00+00:00", "2022-02-22T04:00:00+00:00", "2022-02-22T03:00:00+00:00", "2022-02-22T02:00:00+00:00", "2022-02-22T01:00:00+00:00", "2022-02-22T00:00:00+00:00", "2022-02-21T23:00:00+00:00" ], "Values": [ 91.55364583333333, 55.066631944444445, 2.6114930555555556, 24.9496875, 40.94725694444445, 25.61819444444444, 0.0 ], "StatusCode": "Complete" } ], "Messages": [] } ``` **Note** If you specify a short period and a long time range, you might need to modify the `MaxDatapoints` value, which is by default set to 24 in the script. This represents one data point per hour and 24 per day. ------ #### [ AWS Management Console ] 1. Log into the AWS Management Console and navigate to the CloudWatch service page at [Getting Started with the AWS Management Console](https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/getting-started.html). Select the appropriate AWS Region if necessary. 1. Locate the Metrics section on the left navigation bar and choose **All metrics**. 1. This opens a dashboard with two panels. The top panel shows you the graphic, and the bottom panel has the metrics that you want to graph. Choose the Amazon Keyspaces panel. 1. Choose the **Table Metrics** category from the sub panels. This shows you the tables in your current AWS Region. 1. Identify your table name by scrolling down the menu and selecting the write operation metrics: `ConsumedWriteCapacityUnits` and `ProvisionedWriteCapacityUnits` **Note** This example talks about write operation metrics, but you can also use these steps to graph the read operation metrics. 1. Select the **Graphed metrics (2)** tab to modify the formulas. By default CloudWatch chooses the statistical function **Average** for the graphs. 1. While having both graphed metrics selected (the checkbox on the left) select the menu **Add math**, followed by **Common**, and then select the **Percentage** function. Repeat the procedure twice. First time selecting the **Percentage** function. Second time selecting the **Percentage** function. 1. At this point you should have four metrics in the bottom menu. Let’s work on the `ConsumedWriteCapacityUnits` calculation. To be consistent, you need to match the names with the ones you used in the AWS CLI section. Click on the **m1 ID** and change this value to **consumedWCU**. 1. Change the statistic from **Average** to **Sum**. This action automatically creates another metric called **ANOMALY\$1DETECTION\$1BAND**. For the scope of this procedure, you can ignore this by removing the checkbox on the newly generated **ad1 metric**. 1. Repeat step 8 to rename the **m2 ID** to **provisionedWCU**. Leave the statistic set to **Average**. 1. Choose the **Expression1** label and update the value to **m1** and the label to **Consumed WCUs**. **Note** Make sure you have only selected **m1** (checkbox on the left) and **provisionedWCU** to properly visualize the data. Update the formula by clicking in **Details** and changing the formula to **consumedWCU/PERIOD(consumedWCU)**. This step might also generate another **ANOMALY\$1DETECTION\$1BAND** metric, but for the scope of this procedure you can ignore it. 1. You should now have two graphics: one that indicates your provisioned WCUs on the table and another that indicates the consumed WCUs. 1. Update the percentage formula by selecting the Expression2 graphic (**e2**). Rename the labels and IDs to **utilizationPercentage**. Rename the formula to match **100\$1(m1/provisionedWCU)**. 1. Remove the checkbox from all the metrics except **utilizationPercentage** to visualize your utilization patterns. The default interval is set to 1 minute, but feel free to modify it as needed. The results you get depend on the actual data from your workload. Intervals with more than 100% utilization are prone to low throughput capacity error events. Amazon Keyspaces offers [burst capacity](throughput-bursting.md), but as soon as the burst capacity is exhausted, anything above 100% experiences low throughput capacity error events. ------ ## How to identify under-provisioned Amazon Keyspaces tables For most workloads, a table is considered under-provisioned when it constantly consumes more than 80% of its provisioned capacity. [Burst capacity](throughput-bursting.md) is an Amazon Keyspaces feature that allow customers to temporarily consume more RCUs/WCUs than originally provisioned (more than the per-second provisioned throughput that was defined for the table). The burst capacity was created to absorb sudden increases in traffic due to special events or usage spikes. This burst capacity limited, for more information, see [Use burst capacity effectively in Amazon Keyspaces](throughput-bursting.md). As soon as the unused RCUs and WCUs are depleted, you can experience low capacity throughput error events if you try to consume more capacity than provisioned. When your application traffic is getting close to the 80% utilization rate, your risk of experiencing low capacity throughput error events is significantly higher. The 80% utilization rate rule varies from the seasonality of your data and your traffic growth. Consider the following scenarios: + If your traffic has been **stable** at \$190% utilization rate for the last 12 months, your table has just the right capacity + If your application traffic is **growing** at a rate of 8% monthly in less than 3 months, you will arrive at 100% + If your application traffic is **growing** at a rate of 5% in a little more than 4 months, you will still arrive at 100% The results from the queries above provide a picture of your utilization rate. Use them as a guide to further evaluate other metrics that can help you choose to increase your table capacity as required (for example: a monthly or weekly growth rate). Work with your operations team to define what is a good percentage for your workload and your tables. There are special scenarios where the data is skewed when you analyse it on a daily or weekly basis. For example, with seasonal applications that have spikes in usage during working hours (but then drop to almost zero outside of working hours), you could benefit from [scheduling application auto-scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/examples-scheduled-actions.html), where you specify the hours of the day (and the days of the week) to increase the provisioned capacity, as well as when to reduce it. Instead of aiming for higher capacity so you can cover the busy hours, you can also benefit from [Amazon Keyspaces table auto-scaling](autoscaling.md) configurations if your seasonality is less pronounced. ## How to identify over-provisioned Amazon Keyspaces tables The query results obtained from the scripts above provide the data points required to perform some initial analysis. If your data set presents values lower than 20% utilization for several intervals, your table might be over-provisioned. To further define if you need to reduce the number of WCUs and RCUS, you should revisit the other readings in the intervals. When your table contains several low usage intervals, you can benefit from using Application Auto Scaling policies, either by scheduling Application Auto Scaling or by just configuring the default Application Auto Scaling policies for the table that are based on utilization. If you have a workload with a low utilization to high throttle ratio (**Max(ThrottleEvents)/Min(ThrottleEvents)** in the interval), this could happen when you have a very spiky workload where traffic increases significantly on specific days (or times of day), but is otherwise consistently low. In these scenarios, it might be beneficial to use [scheduled Application Auto Scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/examples-scheduled-actions.html).