In short. Yandex says a changed Yandex AI restriction takes effect after the affected page is recrawled, typically within 2 to 14 days.
Definition: Yandex AI uses pages indexed by Yandex to generate answers. When a page is restricted from Yandex AI with a
robots.txtrule forYandexAdditionalorYandexAdditionalBot, Yandex says the change takes effect after its robots recrawl the page, typically within 2 to 14 days.
Key facts
- Yandex AI answers are generated automatically from pages that Yandex has indexed, according to Yandex's Webmaster guidance for Yandex AI.
- The same guidance names
YandexAdditionalandYandexAdditionalBotas therobots.txtuser agents for a disallow rule that prevents page content from appearing in Yandex AI answers. - Yandex states that the restriction change takes effect when its robots recrawl the page, typically within 2 to 14 days.
- Yandex's Reindex pages documentation says
robots.txtfiles are automatically recrawled within several hours. - For a page submitted through the reindexing tool, Yandex says the indexing bot may take up to three days to visit it, while page data is updated within two weeks after that visit.
What actually has to be recrawled?
Yandex describes two related operations. Its Yandex AI guidance ties the restriction change to a robot recrawl of the page. Its reindexing guidance says the robots.txt file itself is automatically recrawled within several hours. A new rule can therefore be available in the site control file before the affected page is recrawled under that rule.
Yandex AI answers are generated from indexed pages. The restriction is expressed in robots.txt with a disallow directive addressed to one of the two named Yandex AI user agents:
User-agent: YandexAdditionalBot
Disallow: /cats/mycat.html # prevents the content of the page from appearing in Yandex AI answers
The documented 2 to 14 day period is a typical interval for the page recrawl that applies the changed restriction. It is not a timestamp that a site owner can select.
How do the reindexing windows differ?
The Reindex pages tool is for notifying Yandex about new and updated pages. After a page request is submitted, the status documentation says the indexing bot visits the URLs first on its next site crawl, a process that may take up to three days and usually takes less. After the visit, the status called Request processed means that the bot has visited the page, while the data can be updated within two weeks.
Yandex separately says that robots.txt should not be submitted through this tool because those files are automatically recrawled within several hours. The page reindexing window therefore describes an optional notification path for page URLs, while the AI restriction guidance describes the later page recrawl on which the restriction change takes effect.
What should you check on the website?
Yandex's robots.txt documentation says that the file belongs in the site's root directory and must be accessible to indexing bots. It gives an HTTP 200 OK response as one access requirement. The same documentation says Yandex takes the directives in a redirected robots.txt file into account when the target file returns 200 OK.
The robots guidance also separates AI use from removal from Yandex Search. Pages restricted in robots.txt can participate in Yandex Search; for search removal, Yandex directs site owners to use noindex in the HTML or configure the HTTP header. The Yandex AI rule therefore addresses the AI answer use described by that page.
Related terms
FAQ
When should a changed Yandex AI restriction be visible?
Yandex documents a typical 2 to 14 day interval after its robots recrawl the affected page. The robots.txt file itself can be recrawled within several hours.
Can you submit robots.txt through Reindex pages?
Yandex says robots.txt files are automatically recrawled within several hours and should not be manually submitted through the Reindex pages tool. That tool is described for page URLs and other supported documents.
Does the restriction remove a page from Yandex Search?
No. Yandex says pages restricted in robots.txt can participate in Yandex Search. Its robots documentation gives noindex in the HTML or an HTTP header as the route for search removal.
What if the change is still pending after the typical interval?
The Yandex pages opened here give the typical recrawl interval and site access requirements, but they do not describe a dedicated status indicator for Yandex AI restriction processing.
Sources
- Yandex Webmaster, "Search with Yandex AI", n.d.
- Yandex Webmaster, "Reindex pages", n.d.
- Yandex Webmaster, "Using robots.txt", n.d.
Check how a page reads to answer engines. Open the GEO score analyzer