How do AWS WAF Bot Control inspection levels differ?

The Citead team, updated

In short. Common identifies self-identifying bots, while Targeted adds browser, behavior, challenge, and optional machine learning detections.

Definition: AWS WAF Bot Control has Common and Targeted inspection levels. Common identifies and labels self-identifying bots with traditional detection, while Targeted includes Common and adds detection for sophisticated bots that do not self identify.

Key facts

  • AWS describes Common as a basic protection level that labels self-identifying bots, verifies generally desirable bots, and detects high confidence bot signatures in its Bot Control documentation.
  • Targeted adds browser interrogation, fingerprinting, and behavior heuristics for sophisticated bots that do not self identify, according to the same AWS documentation.
  • The AWS WAF Bot Control rule group reference says Targeted combines rate limiting with CAPTCHA and background browser challenges.
  • Targeted machine learning rules use traffic statistics such as timestamps, browser characteristics, and the previous URL visited. AWS says these capabilities are enabled by default and can be disabled in the rule group configuration.

What does Common inspection detect?

Common inspection is designed for bots that identify themselves. AWS lists web scraping frameworks, search engines, and automated browsers as examples, and describes static request data analysis as a traditional detection technique. The rule group labels traffic from detected bots and blocks bots that it cannot verify.

The managed rule reference documents category labels for traffic such as search engines and scraping frameworks. It also describes a verified-bot label for a bot that identifies itself and that Bot Control can verify. A website can therefore distinguish a bot category from its verification status when it evaluates the labels in later AWS WAF rules.

What does Targeted inspection add?

Targeted includes the Common protections and adds detection for sophisticated bots that do not self identify. Its documented techniques include browser interrogation, fingerprinting, and behavior heuristics. AWS documents the targeted response controls as rate limiting, CAPTCHA, and background browser challenges.

Targeted rules use the TGT_ prefix. Rules using machine learning use TGT_ML_; AWS describes them as detecting anomalous behavior that indicates distributed, coordinated bot activity through automated analysis of website traffic statistics. These machine learning capabilities are enabled by default. If they are disabled in the rule group configuration, AWS WAF does not evaluate those rules.

The rule reference also documents targeted signals for token and browser observations. Examples include an automated browser signal, a browser automation extension signal, and browser inconsistency data. Some targeted rules inspect requests with an AWS WAF token, which the same reference describes as part of client session tracking and verification.

TGT_SignalAutomatedBrowser
TGT_SignalBrowserAutomationExtension
TGT_ML_CoordinatedActivityHigh

What can a website do with Bot Control labels?

When AWS WAF evaluates a request against the managed rule group, it can add labels containing the bot category and bot name. AWS says those labels are available to custom rules and are included in CloudWatch metrics and web ACL logs. The managed rule reference gives the Bot Control namespace as awswaf:managed:aws:bot-control: and describes separate namespaces for bot information, targeted details, and signals.

For example, a later rule can match a category or a targeted signal using labels such as these documented forms:

awswaf:managed:aws:bot-control:bot:category:search_engine
awswaf:managed:aws:bot-control:targeted:signal:browser_automation_extension

The level changes the detections and controls supplied by the managed group. Custom AWS WAF rules still determine how a site uses the resulting labels, while the managed rule group records them for monitoring.

FAQ

Does Targeted replace Common inspection?

No. AWS documents Targeted as including the Common-level protections and adding detection for sophisticated bots that do not self identify.

Is machine learning required for Targeted inspection?

Machine learning is an optional configuration within Targeted protection. AWS says the machine learning capabilities are enabled by default, and AWS WAF does not evaluate the TGT_ML_ rules when they are disabled.

Where can I see the bot result?

The managed rule group adds labels to evaluated requests. AWS documents those labels as available to rules that run after the group and as recorded in Amazon CloudWatch metrics.

Sources

  1. AWS, "AWS WAF Bot Control", undated
  2. AWS, "AWS WAF Bot Control rule group", undated

Check how a page reads to answer engines. Open the GEO score analyzer

Test a page and see what to fix first

The analysis usually takes less than 10 seconds and needs no account.

400,000+ pages scored in the public benchmark