**Introducing a new console experience for AWS WAF**
You can now use the updated experience to access AWS WAF functionality anywhere in the console. For more details, see [Working with the console](https://docs.aws.amazon.com/waf/latest/developerguide/working-with-console.html).
# AWS WAF Bot Control
This section explains what Bot Control does.
With Bot Control, you can easily monitor, block, or rate limit bots such as scrapers, scanners, crawlers, status monitors, and search engines. If you use the targeted inspection level of the rule group, you can also challenge bots that don't self identify, making it harder and more expensive for malicious bots to operate against your website. You can protect your applications using the Bot Control managed rule group alone, or in combination with other AWS Managed Rules rule groups and your own custom AWS WAF rules.
Bot Control includes a console dashboard that shows how much of your current traffic is coming from bots, based on request sampling. With the Bot Control managed rule group added to your protection pack (web ACL), you can take action against bot traffic and receive detailed, real-time information about common bot traffic coming to your applications.
**Note**
You are charged additional fees when you use this managed rule group. For more information, see [AWS WAF Pricing](https://aws.amazon.com/waf/pricing/).
The Bot Control managed rule group provides a basic, common protection level that adds labels to self-identifying bots, verifies generally desirable bots, and detects high confidence bot signatures. This gives you the ability to monitor and control common categories of bot traffic.
The Bot Control rule group also provides a targeted protection level that adds detection for sophisticated bots that don't self identify. Targeted protections use detection techniques such as browser interrogation, fingerprinting, and behavior heuristics to identify bad bot traffic. Additionally, targeted protections provide optional automated, machine-learning analysis of website traffic statistics to detect bot-related activity. When you enable machine learning, AWS WAF uses statistics about website traffic, such as timestamps, browser characteristics, and previous URL visited, to improve the Bot Control machine learning model.
When AWS WAF evaluates a web request against the Bot Control managed rule group, the rule group adds labels to requests that it detects as bot related, for example the category of bot and the bot name. You can match against these labels in your own AWS WAF rules to customize handling. The labels that are generated by the Bot Control managed rule group are included in Amazon CloudWatch metrics and your protection pack (web ACL) logs.
You can also use AWS Firewall Manager AWS WAF policies to deploy the Bot Control managed rule group across your applications in multiple accounts that are part of your organization in AWS Organizations.
For more information about the Bot Control managed rule group, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
## Web bot authentication for AI agents
AWS WAF Bot Control now supports Web Bot Authentication (WBA) as a cryptographic verification method for bots and AI agents accessing your CloudFront distributions. This feature enables legitimate AI crawlers and agents to prove their identity without requiring traditional challenge-response mechanisms.
Version requirement: `AWSManagedRulesBotControlRuleSet` Version\$14.0 or later. (The static version must be explicitly selected.) For detailed label taxonomy and rule behavior, see:
+ [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md)
+ [Web request labeling in AWS WAF](waf-labels.md)
+ [AWS Managed Rules changelog](aws-managed-rule-groups-changelog.md)
# AWS WAF Bot Control components
The main components of a Bot Control implementation are the following:
+ **`AWSManagedRulesBotControlRuleSet`** – The Bot Control managed rule group whose rules detect and handle various categories of bots. This rule group add labels to web requests that it detects as bot traffic.
**Note**
You are charged additional fees when you use this managed rule group. For more information, see [AWS WAF Pricing](https://aws.amazon.com/waf/pricing/).
The Bot Control managed rule group provides two levels of protection that you can choose from:
+ **Common** – Detects a variety of self-identifying bots, such as web scraping frameworks, search engines, and automated browsers. Bot Control protections at this level identify common bots using traditional bot detection techniques, such as static request data analysis. The rules label traffic from these bots and block the ones that they cannot verify.
+ **Targeted** – Includes the common-level protections and adds targeted detection for sophisticated bots that do not self identify. Targeted protections mitigate bot activity using a combination of rate limiting and CAPTCHA and background browser challenges.
+ **`TGT_`** – Rules that provide targeted protection have names that begin with `TGT_`. All targeted protections use detection techniques such as browser interrogation, fingerprinting, and behavior heuristics to identify bad bot traffic.
+ **`TGT_ML_`** – Targeted protection rules that use machine learning have names that begin with `TGT_ML_`. These rules use automated, machine-learning analysis of website traffic statistics to detect anomalous behavior indicative of distributed, coordinated bot activity. AWS WAF analyzes statistics about your website traffic such as timestamps, browser characteristics, and previous URL visited, to improve the Bot Control machine learning model. Machine learning capabilities are enabled by default, but you can disable them in your rule group configuration. When machine learning is disabled, AWS WAF does not evaluate these rules.
For details including information about the rule group's rules, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
You include this rule group in your protection pack (web ACL) using a managed rule group reference statement and indicating the inspection level that you want to use. For the targeted level, you also indicate whether to enable machine learning. For more information about adding this managed rule group to your protection pack (web ACL), see [Adding the AWS WAF Bot Control managed rule group to your web ACL](waf-bot-control-rg-using.md).
+ **Bot Control dashboard** – The bot monitoring dashboard for your protection pack (web ACL), available through the protection pack (web ACL) Bot Control tab. Use this dashboard to monitor your traffic and understand how much of it comes from various types of bots. This can be a starting point for customizing your bot management, as described in this topic. You can also use it to verify your changes and monitor activity for various bots and bot categories.
+ **AI Traffic Analysis dashboard** – Specialized dashboard for detailed AI bot and agent activity analysis, available through the protection pack (web ACL) AI Traffic Analysis tab. Provides enhanced visibility into AI-specific traffic patterns, bot intent, and access behaviors beyond standard Bot Control metrics.
+ **JavaScript and mobile application integration SDKs** – You should implement the AWS WAF JavaScript and mobile SDKs if you use the targeted protection level of the Bot Control rule group. The targeted rules use information provided by the SDKs in the client tokens for enhanced detection against malicious bots. For more information about the SDKs, see [Client application integrations in AWS WAF](waf-application-integration.md).
+ **Logging and metrics** – You can monitor your bot traffic and understand how the Bot Control managed rule group evaluates and handles your traffic by studying the data that's collected for your protection pack (web ACL) by AWS WAF logs, Amazon Security Lake, and Amazon CloudWatch. The labels that Bot Control adds to your web requests are included in the data. For information about these options, see [Logging AWS WAF protection pack (web ACL) traffic](logging.md), [Monitoring with Amazon CloudWatch](monitoring-cloudwatch.md), and [What is Amazon Security Lake?](https://docs.aws.amazon.com/security-lake/latest/userguide/what-is-security-lake.html).
Depending on your needs and the traffic that you see, you might want to customize your Bot Control implementation. The following are some of the most commonly used options.
+ **Scope-down statements** – You can exclude some traffic from the web requests that the Bot Control managed rule group evaluates by adding a scope-down statement inside the Bot Control managed rule group reference statement. A scope-down statement can be any nestable rule statement. When a request doesn't match the scope-down statement, AWS WAF evaluates it as not matching the rule group reference statement without evaluating it against the rule group. For more information about scope-down statements, see [Using scope-down statements in AWS WAF](waf-rule-scope-down-statements.md).
Your costs for using the Bot Control managed rule group increase with the number of web requests that AWS WAF evaluates with it. You can help reduce these costs by using a scope-down statement to limit the requests that the rule group evaluates. For example, you might want to allow your homepage to load for everyone, including bots, and then apply the rule group rules to requests that are going to your application APIs or that contain a particular type of content.
+ **Labels and label matching rules** – You can customize how the Bot Control rule group handles some of the bot traffic that it identifies using the AWS WAF label match rule statement. The Bot Control rule group adds labels to your web requests. You can add label matching rules after the Bot Control rule group that match on Bot Control labels and apply the handling that you need. For more information about labeling and using label match statements, see [Label match rule statement](waf-rule-statement-type-label-match.md) and [Web request labeling in AWS WAF](waf-labels.md).
+ **Custom requests and responses** – You can add custom headers to requests that you allow and you can send custom responses for requests that you block by pairing label matching with the AWS WAF custom request and response features. For more information about customizing requests and responses, see [Customized web requests and responses in AWS WAF](waf-custom-request-response.md).
# Using application integration SDKs with Bot Control
This section explains how to use application integration SDKs with Bot Control.
Most of the targeted protections of the Bot Control managed rule group require the challenge tokens that the application integration SDKs generate. The rules that don't require a challenge token on the request are the Bot Control common level protections and the targeted level machine learning rules. For descriptions of the protection levels and rules in the rule group, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
We highly recommend implementing the application integration SDKs, for the most effective use of the Bot Control rule group. The challenge script must run before the Bot Control rule group in order for the rule group to benefit from the tokens that the script acquires.
+ With the application integration SDKs, the script runs automatically.
+ If you're unable to use the SDKs, you can configure your protection pack (web ACL) so that it runs the Challenge or CAPTCHA rule action against all requests that will be inspected by the Bot Control rule group. Using the Challenge or CAPTCHA rule action can incur additional fees. For pricing details, see [AWS WAF Pricing](https://aws.amazon.com/waf/pricing/).
When you implement the application integration SDKs in your clients or use one of the rule actions that runs the challenge script, you expand the capabilities of the rule group and of your overall client application security.
Tokens provide client information with each web request. This additional information enables the Bot Control rule group to separate legitimate client sessions from ill-behaved client sessions, even when both originate from a single IP address. The rule group uses information in the tokens to aggregate client session request behavior for the fine-tuned detection and mitigation that the targeted protections level provide.
For information about the SDKs, see [Client application integrations in AWS WAF](waf-application-integration.md). For information about AWS WAF tokens, see [Token use in AWS WAF intelligent threat mitigation](waf-tokens.md). For information about the rule actions, see [CAPTCHA and Challenge in AWS WAF](waf-captcha-and-challenge.md).
# Adding the AWS WAF Bot Control managed rule group to your web ACL
This section explains how to add and configure the `AWSManagedRulesBotControlRuleSet` rule group.
The Bot Control managed rule group `AWSManagedRulesBotControlRuleSet` requires additional configuration to identify the protection level that you want to implement.
For the rule group description and rules listing, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
This guidance is intended for users who know generally how to create and manage AWS WAF protection packs (web ACLs), rules, and rule groups. Those topics are covered in prior sections of this guide. For basic information about how to add a managed rule group to your protection pack (web ACL), see [Adding a managed rule group to a protection pack (web ACL) through the console](waf-using-managed-rule-group.md).
**Follow best practices**
Use the Bot Control rule group in accordance with the best practices at [Best practices for intelligent threat mitigation in AWS WAF](waf-managed-protections-best-practices.md).
**To use the `AWSManagedRulesBotControlRuleSet` rule group in your protection pack (web ACL)**
1. Add the AWS managed rule group, `AWSManagedRulesBotControlRuleSet` to your protection pack (web ACL). For the full rule group description, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
**Note**
You are charged additional fees when you use this managed rule group. For more information, see [AWS WAF Pricing](https://aws.amazon.com/waf/pricing/).
When you add the rule group, edit it to open the configuration page for the rule group.
1. On the rule group's configuration page, in the **Inspection level** pane, select the inspection level that you want to use.
+ **Common** – Detects a variety of self-identifying bots, such as web scraping frameworks, search engines, and automated browsers. Bot Control protections at this level identify common bots using traditional bot detection techniques, such as static request data analysis. The rules label traffic from these bots and block the ones that they cannot verify.
+ **Targeted** – Includes the common-level protections and adds targeted detection for sophisticated bots that do not self identify. Targeted protections mitigate bot activity using a combination of rate limiting and CAPTCHA and background browser challenges.
+ **`TGT_`** – Rules that provide targeted protection have names that begin with `TGT_`. All targeted protections use detection techniques such as browser interrogation, fingerprinting, and behavior heuristics to identify bad bot traffic.
+ **`TGT_ML_`** – Targeted protection rules that use machine learning have names that begin with `TGT_ML_`. These rules use automated, machine-learning analysis of website traffic statistics to detect anomalous behavior indicative of distributed, coordinated bot activity. AWS WAF analyzes statistics about your website traffic such as timestamps, browser characteristics, and previous URL visited, to improve the Bot Control machine learning model. Machine learning capabilities are enabled by default, but you can disable them in your rule group configuration. When machine learning is disabled, AWS WAF does not evaluate these rules.
1. If you're using the targeted protection level and you don't want AWS WAF to use machine learning (ML) to analyze web traffic for distributed, coordinated bot activity, disable the machine learning option. Machine learning is required for the Bot Control rules whose names start with `TGT_ML_`. For details about these rules, see [Bot Control rules listing](aws-managed-rule-groups-bot.md#aws-managed-rule-groups-bot-rules).
1. Add a scope-down statement for the rule group, to contain the costs of using it. A scope-down statement narrows the set of requests that the rule group inspects. For example use cases, start with [Bot Control example: Using Bot Control only for the login page](waf-bot-control-example-scope-down-login.md) and [Bot Control example: Using Bot Control only for dynamic content](waf-bot-control-example-scope-down-dynamic-content.md).
1. Provide any additional configuration that you need for the rule group.
1. Save your changes to the protection pack (web ACL).
Before you deploy your Bot Control implementation for production traffic, test and tune it in a staging or testing environment until you are comfortable with the potential impact to your traffic. Then test and tune the rules in count mode with your production traffic before enabling them. See the sections that follow for guidance.
# Example scenarios of false positives with AWS WAF Bot Control
This section provides example situations where you might encounter false positives with AWS WAF Bot Control.
We have carefully selected the rules in the AWS WAF Bot Control managed rule group to minimize false positives. We test the rules against global traffic and monitor their impact on test protection packs (web ACLs). However, it's still possible to get false positives due to changes in traffic patterns. Additionally, some use cases are known to cause false positives and will require customization specific to your web traffic.
Situations where you might encounter false positives include the following:
+ Mobile apps typically have non-browser user agents, which the `SignalNonBrowserUserAgent` rule blocks by default. If you expect traffic from mobile apps, or any other legitimate traffic with non-browser user agents, you'll need to add an exception to allow it.
+ You might rely on some specific bot traffic for things like uptime monitoring, integration testing, or marketing tools. If Bot Control identifies and blocks the bot traffic that you want to allow, you need to alter the handling by adding your own rules. While this isn't a false positive scenario for all customers, if it is for you, you will need to handle it the same as for a false positive.
+ The Bot Control managed rule group verifies bots using the IP addresses from AWS WAF. If you use Bot Control and you have verified bots that route through a proxy or load balancer, you might need to explicitly allow them using a custom rule. For information about how to create a custom rule of this type, see [Using forwarded IP addresses in AWS WAF](waf-rule-statement-forwarded-ip-address.md).
+ A Bot Control rule with a low global false positive rate might heavily impact specific devices or applications. For example, in testing and validation, we might not have observed requests from applications with low traffic volumes or from less common browsers or devices.
+ A Bot Control rule that has a historically low false positive rate might have increased false positives for valid traffic. This might be due to new traffic patterns or request attributes that emerge with valid traffic, causing it to match the rule where it didn't before. These changes might be due to situations like the following:
+ Traffic details that are altered as traffic flows through network appliances, such as load balancers or content distribution networks (CDN).
+ Emerging changes in traffic data, for example new browsers or new versions for existing browsers.
For information about how to handle false positives that you might get from the AWS WAF Bot Control managed rule group, see the guidance in the section that follows, [Testing and deploying AWS WAF Bot Control](waf-bot-control-deploying.md).
# Testing and deploying AWS WAF Bot Control
This section provides general guidance for configuring and testing an AWS WAF Bot Control implementation for your site. The specific steps that you choose to follow will depend on your needs, resources, and the web requests that you receive.
This information is in addition to the general information about testing and tuning provided at [Testing and tuning your AWS WAF protections](web-acl-testing.md).
**Note**
AWS Managed Rules are designed to protect you from common web threats. When used in accordance with the documentation, AWS Managed Rules rule groups add another layer of security for your applications. However, AWS Managed Rules rule groups aren't intended as a replacement for your security responsibilities, which are determined by the AWS resources that you select. Refer to the [Shared Responsibility Model](https://aws.amazon.com/compliance/shared-responsibility-model/) to ensure that your resources in AWS are properly protected.
**Production traffic risk**
Before you deploy your Bot Control implementation for production traffic, test and tune it in a staging or testing environment until you are comfortable with the potential impact to your traffic. Then test and tune the rules in count mode with your production traffic before enabling them.
This guidance is intended for users who know generally how to create and manage AWS WAF protection packs (web ACLs), rules, and rule groups. Those topics are covered in prior sections of this guide.
**To configure and test a Bot Control implementation**
Perform these steps first in a test environment, then in production.
1.
**Add the Bot Control managed rule group**
**Note**
You are charged additional fees when you use this managed rule group. For more information, see [AWS WAF Pricing](https://aws.amazon.com/waf/pricing/).
Add the managed AWS rule group `AWSManagedRulesBotControlRuleSet` to a new or existing protection pack (web ACL) and configure it so that it doesn't alter current protection pack (web ACL) behavior.
+ When you add the managed rule group, edit it and do the following:
+ In the **Inspection level** pane, select the inspection level that you want to use.
+ **Common** – Detects a variety of self-identifying bots, such as web scraping frameworks, search engines, and automated browsers. Bot Control protections at this level identify common bots using traditional bot detection techniques, such as static request data analysis. The rules label traffic from these bots and block the ones that they cannot verify.
+ **Targeted** – Includes the common-level protections and adds targeted detection for sophisticated bots that do not self identify. Targeted protections mitigate bot activity using a combination of rate limiting and CAPTCHA and background browser challenges.
+ **`TGT_`** – Rules that provide targeted protection have names that begin with `TGT_`. All targeted protections use detection techniques such as browser interrogation, fingerprinting, and behavior heuristics to identify bad bot traffic.
+ **`TGT_ML_`** – Targeted protection rules that use machine learning have names that begin with `TGT_ML_`. These rules use automated, machine-learning analysis of website traffic statistics to detect anomalous behavior indicative of distributed, coordinated bot activity. AWS WAF analyzes statistics about your website traffic such as timestamps, browser characteristics, and previous URL visited, to improve the Bot Control machine learning model. Machine learning capabilities are enabled by default, but you can disable them in your rule group configuration. When machine learning is disabled, AWS WAF does not evaluate these rules.
For more information about this choice, see [AWS WAF Bot Control rule group](aws-managed-rule-groups-bot.md).
+ In the **Rules** pane, open the **Override all rule actions** dropdown and choose **Count**. With this configuration, AWS WAF evaluates requests against all of the rules in the rule group and only counts the matches that result, while still adding labels to requests. For more information, see [Overriding rule actions in a rule group](web-acl-rule-group-settings.md#web-acl-rule-group-rule-action-override).
With this override, you can monitor the potential impact of the Bot Control rules on your traffic, to determine whether you want to add exceptions for things like internal use cases or desired bots.
+ Position the rule group so that it's evaluated last in the protection pack (web ACL), with a priority setting that's numerically higher than any other rules or rule groups that you're already using. For more information, see [Setting rule priority](web-acl-processing-order.md).
This way, your current handling of traffic isn't disrupted. For example, if you have rules that detect malicious traffic such as SQL injection or cross-site scripting, they'll continue to detect and log those requests. Alternately, if you have rules that allow known non-malicious traffic, they can continue to allow that traffic, without having it blocked by the Bot Control managed rule group. You might decide to adjust the processing order during your testing and tuning activities, but this is a good way to start.
1.
**Enable logging and metrics for the protection pack (web ACL)**
As needed, configure logging, Amazon Security Lake data collection, request sampling, and Amazon CloudWatch metrics for the protection pack (web ACL). You can use these visibility tools to monitor the interaction of the Bot Control managed rule group with your traffic.
+ For information about logging, see [Logging AWS WAF protection pack (web ACL) traffic](logging.md).
+ For information about Amazon Security Lake, see [What is Amazon Security Lake?](https://docs.aws.amazon.com/security-lake/latest/userguide/what-is-security-lake.html) and [Collecting data from AWS services](https://docs.aws.amazon.com/security-lake/latest/userguide/internal-sources.html) in the *Amazon Security Lake user guide*.
+ For information about Amazon CloudWatch metrics, see [Monitoring with Amazon CloudWatch](monitoring-cloudwatch.md).
+ For information about web request sampling, see [Viewing a sample of web requests](web-acl-testing-view-sample.md).
1.
**Associate the protection pack (web ACL) with a resource**
If the protection pack (web ACL) isn't already associated with a resource, associate it. For information, see [Associating or disassociating protection with an AWS resource](web-acl-associating-aws-resource.md).
1.
**Monitor traffic and Bot Control rule matches**
Make sure that traffic is flowing and that the Bot Control managed rule group rules are adding labels to matching web requests. You can see the labels in the logs and see bot and label metrics in the Amazon CloudWatch metrics. In the logs, the rules that you've overridden to count in the rule group show up in the `ruleGroupList` with `action` set to count, and with `overriddenAction` indicating the configured rule action that you overrode.
**Note**
The Bot Control managed rule group verifies bots using the IP addresses from AWS WAF. If you use Bot Control and you have verified bots that route through a proxy or load balancer, you might need to explicitly allow them using a custom rule. For information about how to create a custom rule, see [Using forwarded IP addresses in AWS WAF](waf-rule-statement-forwarded-ip-address.md). For information about how you can use the rule to customize Bot Control web request handling, see the next step.
Carefully review the web request handling for any false positives that you might need to mitigate with custom handling. For examples of false positives, see [Example scenarios of false positives with AWS WAF Bot Control](waf-bot-control-false-positives.md).
1.
**Customize Bot Control web request handling**
As needed, add your own rules that explicitly allow or block requests, to change how Bot Control rules would otherwise handle them.
How you do this depends on your use case, but the following are common solutions:
+ Explicitly allow requests with a rule that you add before the Bot Control managed rule group. With this, the allowed requests never reach the rule group for evaluation. This can help contain the cost of using the Bot Control managed rule group.
+ Exclude requests from Bot Control evaluation by adding a scope-down statement inside the Bot Control managed rule group statement. This functions the same as the preceding option. It can help contain the cost of using the Bot Control managed rule group because the requests that don't match the scope-down statement never reach rule group evaluation. For information about scope-down statements, see [Using scope-down statements in AWS WAF](waf-rule-scope-down-statements.md).
For examples, see the following:
+ [Excluding IP range from bot management](waf-bot-control-example-scope-down-ip.md)
+ [Allowing traffic from a bot that you control](waf-bot-control-example-scope-down-your-bot.md)
+ Use Bot Control labels in request handling to allow or block requests. Add a label match rule after the Bot Control managed rule group to filter out labeled requests that you want to allow from those that you want to block.
After testing, keep the related Bot Control rules in count mode, and maintain the request handling decisions in your custom rule. For information about label match statements, see [Label match rule statement](waf-rule-statement-type-label-match.md).
For examples of this type of customization, see the following:
+ [Creating an exception for a blocked user agent](waf-bot-control-example-user-agent-exception.md)
+ [Allowing a specific blocked bot](waf-bot-control-example-allow-blocked-bot.md)
+ [Blocking verified bots](waf-bot-control-example-block-verified-bots.md)
For additional examples, see [AWS WAF Bot Control examples](waf-bot-control-examples.md).
1.
**As needed, enable the Bot Control managed rule group settings**
Depending on your situation, you might have decided that you want to leave some Bot Control rules in count mode or with a different action override. For the rules that you want to have run as they are configured inside the rule group, enable the regular rule configuration. To do this, edit the rule group statement in your protection pack (web ACL) and make your changes in the **Rules** pane.
# AWS WAF Bot Control examples
This section shows example configurations that satisfy a variety of common use cases for AWS WAF Bot Control implementations.
Each example provides a description of the use case and then shows the solution in JSON listings for the custom configured rules.
**Note**
The JSON listings shown in these examples were created in the console by configuring the rule and then editing it using the **Rule JSON editor**.
**Topics**
+ [Bot Control example: Simple configuration](waf-bot-control-example-basic.md)
+ [Bot Control example: Explicitly allowing verified bots](waf-bot-control-example-allow-verified-bots.md)
+ [Bot Control example: Blocking verified bots](waf-bot-control-example-block-verified-bots.md)
+ [Bot Control example: Allowing a specific blocked bot](waf-bot-control-example-allow-blocked-bot.md)
+ [Bot Control example: Creating an exception for a blocked user agent](waf-bot-control-example-user-agent-exception.md)
+ [Bot Control example: Using Bot Control only for the login page](waf-bot-control-example-scope-down-login.md)
+ [Bot Control example: Using Bot Control only for dynamic content](waf-bot-control-example-scope-down-dynamic-content.md)
+ [Bot Control example: Excluding IP range from bot management](waf-bot-control-example-scope-down-ip.md)
+ [Bot Control example: Allowing traffic from a bot that you control](waf-bot-control-example-scope-down-your-bot.md)
+ [Bot Control example: Enabling targeted inspection level](waf-bot-control-example-targeted-inspection-level.md)
+ [Bot Control example: Using two statements to limit the use of the targeted inspection level](waf-bot-control-example-common-and-targeted-inspection-level.md)
# Bot Control example: Simple configuration
The following JSON listing shows an example protection pack (web ACL) with an AWS WAF Bot Control managed rule group. Note the visibility configuration, which causes AWS WAF to store request samples and metrics for monitoring purposes.
```
{
"Name": "Bot-WebACL",
"Id": "...",
"ARN": "...",
"DefaultAction": {
"Allow": {}
},
"Description": "Bot-WebACL",
"Rules": [
{
...
},
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
}
}
}
],
"VisibilityConfig": {
...
},
"Capacity": 1496,
"ManagedByFirewallManager": false,
"RetrofittedByFirewallManager": false
}
```
# Bot Control example: Explicitly allowing verified bots
AWS WAF Bot Control doesn't block bots that are known by AWS to be common and verifiable bots. When Bot Control identifies a web request as coming from a verified bot, it adds a label that names the bot and a label that indicates that it's a verified bot. Bot Control doesn't add any other labels, such as signals labels, in order to prevent known good bots from being blocked.
You might have other AWS WAF rules that block verified bots. If you want to ensure that verified bots are allowed, add a custom rule to allow them based on the Bot Control labels. Your new rule must run after the Bot Control managed rule group, so that the labels are available to match against.
The following rule explicitly allows verified bots.
```
{
"Name": "match_rule",
"Statement": {
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:verified"
}
},
"RuleLabels": [],
"Action": {
"Allow": {}
}
}
```
# Bot Control example: Blocking verified bots
In order to block verified bots, you must add a rule to block them that runs after the AWS WAF Bot Control managed rule group. To do this, identify the bot names that you want to block and use a label match statement to identify and block them. If you want to just block all verified bots, you can omit the match against the `bot:name:` label.
The following rule blocks only the `bingbot` verified bot. This rule must run after the Bot Control managed rule group.
```
{
"Name": "match_rule",
"Statement": {
"AndStatement": {
"Statements": [
{
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:name:bingbot"
}
},
{
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:verified"
}
}
]
}
},
"RuleLabels": [],
"Action": {
"Block": {}
}
}
```
The following rule blocks all verified bots.
```
{
"Name": "match_rule",
"Statement": {
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:verified"
}
},
"RuleLabels": [],
"Action": {
"Block": {}
}
}
```
# Bot Control example: Allowing a specific blocked bot
It's possible for a bot to be blocked by more than one of the Bot Control rules. Run through the following procedure for each blocking rule.
If an AWS WAF Bot Control rule is blocking a bot that you do not want to block, do the following:
1. Identify the Bot Control rule that's blocking the bot by checking the logs. The blocking rule will be specified in the logs in the fields whose names start with `terminatingRule`. For information about the protection pack (web ACL) logs, see [Logging AWS WAF protection pack (web ACL) traffic](logging.md). Note the label that the rule is adds to the requests.
1. In your protection pack (web ACL), override the action of the blocking rule to count. To do this in the console, edit the rule group rule in the protection pack (web ACL) and choose a rule action override of Count for the rule. This ensures that the bot is not blocked by the rule, but the rule will still apply its label to matching requests.
1. Add a label matching rule to your protection pack (web ACL), after the Bot Control managed rule group. Configure the rule to match against the overridden rule's label and to block all matching requests except for the bot that you don't want to block.
Your protection pack (web ACL) is now configured so that the bot you want to allow is no longer blocked by the blocking rule that you identified through the logs.
Check traffic and your logs again, to be sure that the bot is being allowed through. If not, run through the above procedure again.
For example, suppose you want to block all monitoring bots except for `pingdom`. In this case, you override the `CategoryMonitoring` rule to count and then write a rule to block all monitoring bots except for those with the bot name label `pingdom`.
The following rule uses the Bot Control managed rule group but overrides the rule action for `CategoryMonitoring` to count. The category monitoring rule applies its labels as usual to matching requests, but only counts them instead of performing its usual action of block.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [
{
"ActionToUse": {
"Count": {}
},
"Name": "CategoryMonitoring"
}
],
"ExcludedRules": []
}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
}
}
```
The following rule matches against the category monitoring label that the preceding `CategoryMonitoring` rule adds to matching web requests. Among the category monitoring requests, this rule blocks all but those that have a label for the bot name `pingdom`.
The following rule must run after the preceding Bot Control managed rule group in the protection pack (web ACL) processing order.
```
{
"Name": "match_rule",
"Priority": 10,
"Statement": {
"AndStatement": {
"Statements": [
{
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:category:monitoring"
}
},
{
"NotStatement": {
"Statement": {
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:bot:name:pingdom"
}
}
}
}
]
}
},
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "match_rule"
}
}
```
# Bot Control example: Creating an exception for a blocked user agent
If traffic from some non-browser user agents is being erroneously blocked, you can create an exception by setting the offending AWS WAF Bot Control rule `SignalNonBrowserUserAgent` to Count and then combining the rule's labeling with your exception criteria.
**Note**
Mobile apps typically have non-browser user agents, which the `SignalNonBrowserUserAgent` rule blocks by default.
The following rule uses the Bot Control managed rule group but overrides the rule action for `SignalNonBrowserUserAgent` to Count. The signal rule applies its labels as usual to matching requests, but only counts them instead of performing its usual action of block.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [
{
"ActionToUse": {
"Count": {}
},
"Name": "SignalNonBrowserUserAgent"
}
],
"ExcludedRules": []
}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
}
}
```
The following rule matches against the signal label that the Bot Control `SignalNonBrowserUserAgent` rule adds to its matching web requests. Among the signal requests, this rule blocks all but those that have the user agent that we want to allow.
The following rule must run after the preceding Bot Control managed rule group in the protection pack (web ACL) processing order.
```
{
"Name": "match_rule",
"Statement": {
"AndStatement": {
"Statements": [
{
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:bot-control:signal:non_browser_user_agent"
}
},
{
"NotStatement": {
"Statement": {
"ByteMatchStatement": {
"FieldToMatch": {
"SingleHeader": {
"Name": "user-agent"
}
},
"PositionalConstraint": "EXACTLY",
"SearchString": "PostmanRuntime/7.29.2",
"TextTransformations": [
{
"Priority": 0,
"Type": "NONE"
}
]
}
}
}
}
]
}
},
"RuleLabels": [],
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "match_rule"
}
}
```
# Bot Control example: Using Bot Control only for the login page
The following example uses a scope-down statement to apply AWS WAF Bot Control only for traffic that's coming to a website's login page, which is identified by the URI path `login`. The URI path to your login page might be different from the example, depending on your application and environment.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
},
"ScopeDownStatement": {
"ByteMatchStatement": {
"SearchString": "login",
"FieldToMatch": {
"UriPath": {}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "NONE"
}
],
"PositionalConstraint": "CONTAINS"
}
}
}
}
```
# Bot Control example: Using Bot Control only for dynamic content
This example uses a scope-down statement to apply AWS WAF Bot Control only to dynamic content.
The scope-down statement excludes static content by negating the match results for a regex pattern set:
+ The regex pattern set is configured to match extensions of *static content*. For example, the regex pattern set specification might be `(?i)\.(jpe?g|gif|png|svg|ico|css|js|woff2?)$`. For information about regex pattern sets and statements, see [Regex pattern set match rule statement](waf-rule-statement-type-regex-pattern-set-match.md).
+ In the scope-down statement, we exclude the matching static content by nesting the regex pattern set statement inside a `NOT` statement. For information about the `NOT` statement, see [NOT rule statement](waf-rule-statement-type-not.md).
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
},
"ScopeDownStatement": {
"NotStatement": {
"Statement": {
"RegexPatternSetReferenceStatement": {
"ARN": "arn:aws:wafv2:us-east-1:123456789:regional/regexpatternset/excludeset/00000000-0000-0000-0000-000000000000",
"FieldToMatch": {
"UriPath": {}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "NONE"
}
]
}
}
}
}
}
}
```
# Bot Control example: Excluding IP range from bot management
If you want to exclude a subset of web traffic from AWS WAF Bot Control management, and you can identify that subset using a rule statement, then exclude it by adding a scope-down statement to your Bot Control managed rule group statement.
The following rule performs normal Bot Control bot management on all web traffic except for web requests coming from a specific IP address range.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
},
"ScopeDownStatement": {
"NotStatement": {
"Statement": {
"IPSetReferenceStatement": {
"ARN": "arn:aws:wafv2:us-east-1:123456789:regional/ipset/friendlyips/00000000-0000-0000-0000-000000000000"
}
}
}
}
}
}
```
# Bot Control example: Allowing traffic from a bot that you control
You can configure some site monitoring bots and custom bots to send custom headers. If you want to allow traffic from these types of bots, you can configure them to add a shared secret in a header. You then can exclude messages that have the header by adding a scope-down statement to the AWS WAF Bot Control managed rule group statement.
The following example rule excludes traffic with a secret header from Bot Control inspection.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
},
"ScopeDownStatement": {
"NotStatement": {
"Statement": {
"ByteMatchStatement": {
"SearchString": "YSBzZWNyZXQ=",
"FieldToMatch": {
"SingleHeader": {
"Name": "x-bypass-secret"
}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "NONE"
}
],
"PositionalConstraint": "EXACTLY"
}
}
}
}
}
}
```
# Bot Control example: Enabling targeted inspection level
For an enhanced level of protection, you can enable the targeted inspection level in your AWS WAF Bot Control managed rule group.
In the following example, machine learning features are enabled. You can opt out of this behavior by setting `EnableMachineLearning` to `false`.
```
{
"Name": "AWS-AWSBotControl-Example",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "TARGETED",
"EnableMachineLearning": true
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Example"
}
}
}
```
# Bot Control example: Using two statements to limit the use of the targeted inspection level
As a cost optimization, you can use two AWS WAF Bot Control managed rule group statements in your protection pack (web ACL), with separate inspection levels and scoping. For instance, you could scope the targeted inspection level statement only to more sensitive application endpoints.
The two statements in the following example have mutually exclusive scoping. Without this configuration, a request could result in two billed Bot Control evaluations.
**Note**
Multiple statements referencing `AWSManagedRulesBotControlRuleSet` are not supported in the visual editor in the console. Instead, use the JSON editor.
```
{
"Name": "Bot-WebACL",
"Id": "...",
"ARN": "...",
"DefaultAction": {
"Allow": {}
},
"Description": "Bot-WebACL",
"Rules": [
{
...
},
{
"Name": "AWS-AWSBotControl-Common",
"Priority": 5,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "COMMON"
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Common"
},
"ScopeDownStatement": {
"NotStatement": {
"Statement": {
"ByteMatchStatement": {
"FieldToMatch": {
"UriPath": {}
},
"PositionalConstraint": "STARTS_WITH",
"SearchString": "/sensitive-endpoint",
"TextTransformations": [
{
"Type": "NONE",
"Priority": 0
}
]
}
}
}
}
}
},
{
"Name": "AWS-AWSBotControl-Targeted",
"Priority": 6,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesBotControlRuleSet",
"ManagedRuleGroupConfigs": [
{
"AWSManagedRulesBotControlRuleSet": {
"InspectionLevel": "TARGETED",
"EnableMachineLearning": true
}
}
],
"RuleActionOverrides": [],
"ExcludedRules": []
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSBotControl-Targeted"
},
"ScopeDownStatement": {
"Statement": {
"ByteMatchStatement": {
"FieldToMatch": {
"UriPath": {}
},
"PositionalConstraint": "STARTS_WITH",
"SearchString": "/sensitive-endpoint",
"TextTransformations": [
{
"Type": "NONE",
"Priority": 0
}
]
}
}
}
}
}
],
"VisibilityConfig": {
...
},
"Capacity": 1496,
"ManagedByFirewallManager": false,
"RetrofittedByFirewallManager": false
}
```