In short. A YandexFeed visit records a structured-data fetch and does not show that a page entered Yandex search.
Definition: No. YandexFeed is a Yandex robot that downloads structured data through links submitted by website owners to Yandex services. Yandex documents that it does not make indexing requests, so a YandexFeed request is evidence of a submitted-data fetch rather than search inclusion.
Key facts
- Yandex identifies
YandexFeed/1.0as the robot that downloads structured data through links submitted to Yandex services. Its purpose is documented in How to check that a robot belongs to Yandex. - The Yandex robot reference marks YandexFeed as a robot that does not make indexing requests.
- Yandex says that a number of its robots download web documents for purposes other than indexing, and that a document unavailable to the main Yandex robot will never be indexed or found in search results.
- Yandex describes indexing as saving pages to its database and processing them with algorithms. Search results are generated from the uploaded data, as explained in How does Yandex search work?.
What does a YandexFeed request show?
The request shows that a Yandex service has followed a submitted link to retrieve structured data. It identifies the fetcher, its stated purpose, and the URL that received the request. It does not show that the main indexing robot has crawled the page or that the page is in the search database.
The complete user-agent string documented for this robot is:
Mozilla/5.0 (compatible; YandexFeed/1.0; +http://yandex.com/bots)
Yandex groups this robot with other robots that download documents for purposes other than indexing. Its table separately describes YandexBot/3.0 as the main indexing robot. A YandexFeed visit can therefore coexist with no YandexBot visit to the same URL.
How is search inclusion decided?
Yandex documents several stages. During crawling, the robot determines which sites to crawl and how often. The known-page list can come from internal and external links, a Sitemap file, Yandex Metrica data, and robots.txt. The documentation says that pages larger than 10 MB are not indexed.
In the loading and processing stage, the robot determines page content and saves it to its database. Yandex lists the description meta tag, the title element, and Schema.org markup as examples of content it analyzes. It also says that a noindex directive in the robots meta tag means the page will not be included in search results.
The following stage is a separate algorithmic decision. Yandex says that algorithms determine which pages can be included based on the information collected by the robot and various ranking and indexing factors. It gives duplicate pages and pages with indexing disabled as examples that the database will not include. It also states that a page can contain original, structured text and still not be added to the database.
How can you check the visit and the page status?
Start with server logs. Yandex recommends a reverse DNS lookup for the requesting IP address, followed by a forward lookup to confirm that the hostname resolves to the same IP address. Yandex says its robot hostnames end in yandex.ru, yandex.net, or yandex.com; a different ending indicates that the robot does not belong to Yandex. This verification procedure is described in How to check that a robot belongs to Yandex.
For page status, Yandex Webmaster's Page check reports the response code, how the content looks to the bot, and whether the page is included in or excluded from search. The tool also lets you select a bot and displays its user-agent. A response code different from 200 OK may lead to exclusion, according to Page check.
The robot reference marks YandexFeed as ignoring the general rules in robots.txt. Yandex's User-agent documentation says that when a Yandex robot may ignore general directives, you should specify that robot's exact user-agent. A matching rule is:
User-agent: YandexFeed
Disallow: /
This controls the documented YandexFeed fetch separately from the main indexing robot's access.
Related terms
FAQ
Can YandexFeed add a URL to Yandex search?
Yandex documents YandexFeed as a structured-data fetcher and says it does not make indexing requests. Search inclusion is decided through the separate indexing workflow.
Does a YandexFeed request prove that YandexBot can access the page?
No. Yandex explicitly states that if a special-purpose robot downloads a document that the main Yandex robot cannot access, the document will never be indexed or found in search results.
Where can you confirm whether the page is in search?
Use Yandex Webmaster's Page check. It reports page status and lets you select a bot to inspect access, while server logs show the user-agent that made the request.
Sources
- Yandex Webmaster, "How to check that a robot belongs to Yandex",
- Yandex Webmaster, "How does Yandex search work?",
- Yandex Webmaster, "The User-agent directive",
- Yandex Webmaster, "Page check",
Check how a page reads to answer engines. Open the GEO score analyzer