QwantBot

The Citead team, updated

In short. QwantBot is Qwant's search crawler, with documented request verification and robots.txt controls for access and crawl timing.

Definition: QwantBot is the crawler that maintains Qwant's search index. A crawler is an automated client that visits web resources.

Key facts

  • As of October 2026, Qwant's crawler documentation identifies requests by the keyword Qwantbot.
  • The IETF's Robots Exclusion Protocol specifies /robots.txt as the location for crawl rules, served as UTF-8 plain text.
  • Qwant's indexing documentation describes quality evaluation after crawling; pages that pass become candidates for search results.

How is a request identified and verified?

A User-Agent is the identification string a client sends in a request. Qwant's identification guidance gives this pattern:

Mozilla/5.0 (compatible; Qwantbot{-news}/X.Y_{worker_id}; +https://help.qwant.com/bot/)

Braces mark optional parts: -news identifies Qwant News, and worker_id identifies the requesting machine.

The same guidance recommends reverse DNS lookup: the requesting IP address resolves to a hostname ending in qwant.com. Forward lookup checks that this hostname returns the original address.

Which robots.txt controls does Qwant document?

A robots.txt file groups crawl rules under a User-agent line. Qwant's access controls include these separate configurations.

Allow crawling throughout the site:

User-agent: Qwantbot
Allow: /

Block one directory:

User-agent: Qwantbot
Disallow: /prive/

Block the whole site:

User-agent: Qwantbot
Disallow: /

Set a minimum interval of eight seconds between requests:

User-agent: Qwantbot
Crawl-delay: 8

Each subdomain needs its own file. A file on example.com does not cover blog.example.com.

The Robots Exclusion Protocol defines case-insensitive matching for crawler names. Its fallback rule uses the User-agent: * group when no named group matches. These details matter when reviewing an existing file before adding a Qwantbot group. The protocol also explains that listed paths are public; sensitive resources require access protection such as HTTP authentication.

How does crawling relate to indexing?

Qwant's indexing explanation separates finding and downloading pages from evaluating them for inclusion in its index. Links from known pages and sitemap files supply page addresses. The crawler downloads accessible HTML, then the indexing stage applies quality criteria.

The same documentation excludes password-protected pages, pages blocked through robots.txt, and pages carrying a noindex instruction. Its publisher guidance recommends an up-to-date sitemap.xml and clearly structured HTML.

FAQ

Can I stop indexing with a page-level instruction?

Qwant's indexing documentation lists this HTML instruction among its exclusions:

<meta name="robots" content="noindex">

You can inspect the page's HTML for this tag when checking an indexing problem.

Does robots.txt keep a directory private?

The IETF's security considerations explain that robots.txt exposes listed paths publicly. You need an access control mechanism, such as HTTP authentication, to restrict access to protected content.

Sources

  1. Qwant, "The Qwant bot (QwantBot) and the robots.txt file", 5 October 2026
  2. IETF, "RFC 9309: Robots Exclusion Protocol", September 2022
  3. Qwant, "How Qwant Indexes the Web", 21 July 2026

Check how a page reads to answer engines. Open the GEO score analyzer

Test a page and see what to fix first

The analysis usually takes less than 10 seconds and needs no account.

400,000+ pages scored in the public benchmark