Does YandexAdditionalBot make ordinary indexing requests?

The Citead team, updated

In short. YandexAdditionalBot processes robots.txt for Yandex AI content controls on pages already indexed by Yandex.

Definition: No. Yandex documents YandexAdditionalBot as a robot that processes robots.txt to control whether page content appears in Search with Yandex AI responses. It applies to pages already indexed by the primary crawler and does not make indexing requests.

Key facts

  • Yandex's robot reference lists YandexAdditionalBot and YandexAdditional user-agent forms for the same purpose.
  • The same table says the robot helps process robots.txt, applies to pages indexed by the primary crawler, and does not make indexing requests.
  • Yandex's Search with Yandex AI documentation says answers are generated from pages indexed by Yandex.
  • That documentation names YandexAdditional and YandexAdditionalBot as the user-agent values for a Disallow directive that controls use in Yandex AI answers.
  • Yandex's robots.txt guide defines User-agent as the robot selector and Disallow as a restriction on crawling sections or individual pages.

What does YandexAdditionalBot do?

The answer is no: Yandex's robot reference classifies this robot as a content-control robot rather than an ordinary indexing robot. The reference places it alongside robots that download web documents for purposes other than indexing. Its table identifies YandexBot as the main indexing robot, while the YandexAdditionalBot row says the robot processes robots.txt to prevent page content from appearing in Search with Yandex AI responses.

The reference lists several equivalent user-agent forms in that row:

YandexAdditionalBot
YandexAdditionalBot/1.0
Mozilla/5.0 (compatible; YandexAdditionalBot/1.0; +http://yandex.com/bots)

It also lists YandexAdditional, YandexAdditional/1.0, and the corresponding full browser string. These names identify the robot in server logs and in a robots.txt group.

How does the AI content-control processing work?

Yandex's Search with Yandex AI documentation describes Yandex AI as a search feature that offers a concise answer. It scans the internet for relevant information, analyzes sources in real time, and combines findings into an answer with references. The integration documentation says those answers are generated automatically from pages already indexed by Yandex.

YandexAdditionalBot therefore operates at the boundary between an indexed page and its possible use in a Yandex AI answer. The robot reference says its scope is pages indexed by the primary crawler. The same reference says it does not make indexing requests, so a request from this user-agent is evidence of the control check described by Yandex, rather than a request to add the page to the search index.

How can a site control its use of a page?

Yandex's AI content-control guidance tells site owners to add a Disallow directive for either YandexAdditional or YandexAdditionalBot when they do not want site information used in Yandex AI answers. Its example is:

User-agent: YandexAdditionalBot
Disallow: /cats/mycat.html # prevents the content of the page from appearing in Yandex AI answers

The robots.txt guide says User-agent indicates the robot to which a group applies, while Disallow prohibits crawling of sections or individual pages. Yandex's AI documentation says the change takes effect when Yandex robots recrawl the page, typically within 2 to 14 days.

This control has a separate search-index effect. Yandex's robots.txt guide says pages restricted in robots.txt can participate in Yandex search. To remove a page from search, that guide directs site owners to use the noindex directive in the HTML or configure the HTTP header, rather than relying on a crawl restriction.

How can a site identify the request?

A user-agent string alone is not an authenticity check. Yandex's verification procedure says robots can disguise themselves by indicating a relevant User Agent. It starts with the request IP address in server logs, uses reverse DNS to find the hostname, checks for a hostname ending in yandex.ru, yandex.net, or yandex.com, and then uses forward DNS to confirm the same IP address.

The reference also says Yandex robots use an offline network and IP addresses that change frequently, so their list is not disclosed. For this robot, the documented user-agent forms and the DNS procedure are the available checks in the referenced documentation.

FAQ

Is YandexAdditionalBot needed to get a page indexed?

No. Yandex's robot reference says that YandexAdditionalBot does not make indexing requests and applies to pages indexed by the primary crawler. Yandex AI answers are generated from pages that Yandex has already indexed.

Which user-agent should a site owner target?

Yandex documents both YandexAdditional and YandexAdditionalBot as values for the User-agent group. Its example uses YandexAdditionalBot, followed by a path-specific Disallow rule.

No. Yandex's robots.txt guide says a page restricted in robots.txt can participate in Yandex search. The guide points to noindex in HTML or an HTTP header for removing a page from search.

Sources

  1. Yandex Webmaster, "How to check that a robot belongs to Yandex", undated
  2. Yandex Webmaster, "Search with Yandex AI", undated
  3. Yandex Webmaster, "Using robots.txt", undated

Check how a page reads to answer engines. Open the GEO score analyzer

Test a page and see what to fix first

The analysis usually takes less than 10 seconds and needs no account.

400,000+ pages scored in the public benchmark