COST01-BP03 Establish cloud budgets and forecasts
Adjust existing organizational budgeting and forecasting processes to be compatible with the highly variable nature of cloud costs and usage. Processes must be dynamic, using trend-based or business driver-based algorithms or a combination of both.
Level of risk exposed if this best practice is not established: High
Implementation guidance
In traditional on-premises IT setups, customers often face the challenge of planning for fixed costs that change only occasionally, typically with the purchase of new IT hardware and services to meet peak demand. In contrast, AWS Cloud adopts a different approach, where customers pay for the resources they use as dictated by their actual IT and business needs. In the cloud environment, demand can fluctuate on a monthly, daily, or even hourly basis.
Using the cloud brings efficiency, speed, and agility, which results in a highly-variable cost and usage pattern. Costs can decrease or sometimes increase in response to greater workload efficiency or the deployment of new workloads and features. As workloads scale to serve an expanding customer base, cloud usage and costs correspondingly rise due to the increased accessibility of resources. This flexibility in cloud services extends to the costs and forecasts, which creates a degree of elasticity.
It's essential to align closely with these shifting business needs and demand drivers, and aim for the most accurate planning possible. Traditional organizational budget processes need to adapt to accommodate this variability.
Consider cost modelling while you forecast the cost for new workloads. Cost modelling creates a baseline understanding of expected cloud costs, which helps you perform total cost of ownership (TCO), return on investment (ROI), and other financial analysis, set targets and expectations with stakeholders, and identify opportunities for cost optimization.
Your organization should understand the cost definitions and accepted groupings. The level of detail at which you forecast can vary based on your organization's structure and internal workflows. Select a granularity that suits your specific requirements and organizational setup. It is important to understand at what level the forecast is performed:
-
Management account or AWS Organizations level: The management account is the account that you use to create AWS Organizations. Organizations have one management account by default.
-
Linked or member account: An account in Organizations is a standard AWS account that contains your AWS resources and the identities that can access those resources.
-
Environment: An environment is a collection of AWS resources that runs an application version. An environment can be made with multiple linked or member accounts.
-
Project: A project is a combination of set objectives or tasks to be accomplished within a fixed period. It is important to consider the project lifecycle during your forecast.
-
AWS services: Groups or categories such as compute or storage services where you can group AWS services for your forecast.
-
Custom grouping: You can create custom groups based on your organization's needs, such as business units, cost centers, teams, cost allocation tags, cost categories, linked accounts, or combination of these.
Identify the business drivers that can impact your usage cost, and forecast for each of them separately to calculate expected usage in advance. Some of the drivers might be linked to IT and product teams within the organization. Other business drivers, such as marketing events, promotions, geographic expansions, mergers, and acquisitions, are known by your sales, marketing, and business leaders, and it's important to collaborate and account for all those demand drivers as well.
You can use AWS Cost Explorer for trend-based forecasting in a defined future time range based on your past spend. AWS Cost Explorer's forecasting engine segments your historical data based on charge types (for example, Reserved Instances) and uses a combination of machine learning and rule-based models to predict spend across all charge types individually.
Once you've established your forecast process and built models,
you can
use AWS Budgets
To stay informed on the performance of your existing budgets, you can create and schedule AWS Budgets reports to email you and your stakeholders on a regular cadence. You can also create AWS Budgets alerts based on actual costs, which are reactive in nature, or on forecasted costs, which provides time to implement mitigations against potential cost overruns. You can be alerted when your cost or usage actually exceeds a certain level or if they are forecasted to exceed your budgeted amount.
Adjust existing budget and forecast processes to be more dynamic
using trend-based algorithms (with historical costs as inputs) and
driver-based algorithms (for example, new product launches,
Regional expansion, or new environments for workloads), which are
ideal for a dynamic and variable spending environment. Once you've
determined your trend-based forecast using Cost Explorer or any
other tools, use
the AWS Pricing Calculator
Track the accuracy of that forecast, as budgets should be set based on these forecast calculations and estimations. Monitor the accuracy and effectiveness of the integrated cloud cost forecasts. Regularly review actual spend compared to your forecast, and adjust as needed to improve forecast precision. Track forecast variance, and perform root cause analysis on reported variance to act and adjust forecasts.
As mentioned in the COST01-BP02 Establish a partnership between finance and technology, it is important to foster a partnership and cadence between IT, finance, and other stakeholders to verify that they are all using the same tools or processes for consistency. In cases where budgets may need to change, increase cadence touch points to react to those changes more quickly.
Implementation steps
-
Define the cost language within the organization: Create a common AWS cost language within the Organization with multiple dimension and grouping. Make sure stakeholders understand forecast granularity, pricing models, and the level of your cost forecasts.
-
Analyze trend-based forecasts: Use trend-based forecast tools such as AWS Cost Explorer and Amazon Forecast. Analyze your usage cost on multiple dimensions like service, account, tags, and cost categories.
-
Analyze driver-based forecasts: Identify the impact of business drivers on your cloud usage, and forecast for each of them separately to calculate expected usage cost in advance. Work closely with business unit owners and stakeholders to understand the impact on new drivers, and calculate expected cost changes to define accurate budgets.
-
Update existing forecast and budget processes: Based on adopted forecast methods such as trend-based, business driver-based, or a combination of both forecasting methods, define your forecast and budget processes. Budgets should be calculated, realistic, and based on your forecasts.
-
Configure alerts and notifications: Use AWS Budgets alerts and cost anomaly detection to get alerts and notifications.
-
Perform regular reviews with key stakeholders: For example, align on changes in business direction and usage with stakeholders in IT, finance, platform teams, and other areas of the business.
Resources
Related documents:
Related videos:
Related examples: