In short. A website can disallow DuckAssistBot and allow DuckDuckBot through separate robots.txt groups.
Definition: Yes. As of 17 October 2026, DuckDuckGo documents DuckAssistBot for fetching pages for AI-assisted answers and DuckDuckBot for search crawling. Separate robots.txt groups let a website disallow DuckAssistBot while allowing DuckDuckBot.
Key facts
- DuckDuckGo's DuckAssistBot help page describes real-time crawling for AI-assisted answers and says the collected data is not used to train AI models.
- DuckDuckGo's DuckDuckBot help page describes a crawler that improves search results and respects WWW::RobotRules, a robots exclusion rules system.
- The IETF's Robots Exclusion Protocol defines a product token as a crawler's name for selecting the applicable robots.txt group.
Which robots.txt rules separate the bots?
A robots.txt file contains crawling instructions grouped by crawler name. The bot names are DuckAssistBot and DuckDuckBot. Applying the User-agent, Disallow and Allow syntax in RFC 9309 gives this configuration:
User-agent: DuckAssistBot
Disallow: /
User-agent: DuckDuckBot
Allow: /
The slash matches paths from the site root. The DuckAssistBot group disallows crawling across those paths; the DuckDuckBot group allows it. The standard places the file at /robots.txt in the service's top-level path and requires UTF-8 encoding with the text/plain media type.
How do existing groups affect this configuration?
RFC 9309's group-selection rules require case-insensitive matching of the crawler's product token. A group names its crawler with a User-agent line, followed by its rules. Placing both bot names above the same Disallow line would apply that restriction to both crawlers.
When several groups match the same token, their rules combine. Adding another DuckDuckBot group therefore requires reviewing existing DuckDuckBot rules: a more specific path restriction can still apply alongside Allow: /. The standard selects the longest matching path and prefers Allow when equivalent allow and disallow rules compete.
The wildcard group, User-agent: *, applies when no named group matches. Consequently, adding a named DuckDuckBot group changes how wildcard restrictions apply to that crawler. Any intended path restrictions need to be included in its named group.
What does the DuckAssistBot opt-out change?
DuckDuckGo's opt-out instructions describe disallowing DuckAssistBot as opting out of being a potential source for its AI-assisted answers. The documented delay is 72 hours after the robots.txt update, after which DuckAssistBot stops crawling the site.
The same documentation states that opting out leaves organic search rankings and whether websites appear in search results unaffected. It provides crawling@duckduckgo.com for opt-out problems.
Related terms
FAQ
Should I put the full browser-style user-agent string in robots.txt?
Use the product token, DuckAssistBot or DuckDuckBot, in the User-agent line. RFC 9309 distinguishes this token from the longer identification string sent in an HTTP request.
Is this an AI model training opt-out?
DuckDuckGo describes DuckAssistBot as fetching pages for AI-assisted answers and says its data is not used to train AI models. Blocking this bot controls that documented answer-fetching activity.
Sources
- DuckDuckGo, "Is DuckAssistBot related to DuckDuckGo?", undated
- DuckDuckGo, "Is DuckDuckBot related to DuckDuckGo?", undated
- IETF, "Robots Exclusion Protocol", September 2022
Check how a page reads to answer engines. Open the GEO score analyzer