In short. MistralAI-Training collects web content for model-training datasets and has a dedicated robots.txt opt-out.

Definition: As of 17 October 2026, MistralAI-Training is Mistral's web crawler for collecting content to help build model-training datasets. Website owners can disallow this crawler through robots.txt, the file that publishes crawler access rules.

Key facts

Mistral's crawler documentation assigns MistralAI-Training to training collection and provides a robots.txt opt-out.
The Robots Exclusion Protocol specifies a lowercase /robots.txt file at the service's top-level path, encoded as UTF-8 and served as text/plain.
The HTTP standard's User-Agent definition describes a request header containing information about the software making the request, with product names and optional versions.

What does MistralAI-Training collect?

Mistral's crawler documentation describes collection of web content to help build datasets for training Mistral models. It assigns automated search indexing to the separate MistralAI-Index crawler. MistralAI-Training does not perform search indexing or answer live user queries in Vibe.

This distinction gives the robots.txt token a specific purpose: controlling Mistral's training crawler. A rule addressed to MistralAI-Training targets that crawler's collection activity.

How does the robots.txt control work?

Mistral's documented opt-out uses the crawler's name in robots.txt. The following configuration expresses a site-wide disallow rule:

User-agent: MistralAI-Training
Disallow: /

In the Robots Exclusion Protocol, the product token is the crawler name used to select a group of rules. Here, User-agent selects MistralAI-Training, and Disallow: / matches paths beginning at the root.

Group selection ignores letter case. Multiple groups matching the same token are combined, so an existing group for MistralAI-Training also needs review. Path rules use the most specific matching path; an existing, more specific Allow rule can create an exception to the root disallow.

These are the standard's matching rules. They explain how to read the configuration alongside Mistral's published opt-out.

How can the configuration be checked?

The robots standard places the file at /robots.txt. Reviewing that publicly served file establishes whether the intended group is present. Rules placed before any User-agent group should be ignored under the standard, so the group heading matters.

For request identification, Mistral's published crawler string contains this versioned identifier:

MistralAI-Training/1.0

The robots.txt group uses MistralAI-Training without the version suffix. The standard describes the robots product token as a substring of the request's identification string.

FAQ

Can I block training collection while keeping separate search controls?

Yes. Mistral's crawler documentation gives training and search indexing separate names: MistralAI-Training and MistralAI-Index. You can address the training token in its own robots.txt group.

Does robots.txt protect private content?

The robots standard recommends application-level security measures, including HTTP authentication, for controlling access. You should use such protection for restricted content; robots.txt publishes crawler instructions publicly.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer