In short. KagiBot respects robots.txt rules targeting Kagibot and follows Googlebot directives when KagiBot rules are absent.

Definition: KagiBot is the web crawler for the Kagi search engine. As of 17 October 2026, Kagi documents a Kagibot request identifier, source IP addresses, and robots.txt handling that includes following Googlebot directives when KagiBot rules are absent.

Key facts

Kagi's bot documentation publishes a Kagibot/1.0 identifier and four source IP addresses. It states that KagiBot respects rules targeting Kagibot and follows Googlebot directives when KagiBot rules are absent.
The IETF's HTTP Semantics specification defines User-Agent as a request header containing information about the requesting software.
The IETF's Robots Exclusion Protocol places crawling rules in /robots.txt at the top-level path of the service.

How are KagiBot requests identified?

A User-Agent header is the request field in which a crawler supplies its identification string. Kagi's About KagiBot page publishes this value:

Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)

The same page lists these IP address and hostname pairs:

216.18.205.234 amd.kagibot.org
35.212.27.76 donna.kagibot.org
104.254.65.50 knet.kagibot.org
209.151.156.194 ogi.kagibot.org

For a website reviewing requests labelled Kagibot, these published values provide a reference for comparing the recorded request header and source address. Kagi also gives vlad@kagi.com as the contact for questions or suspected crawler misbehaviour.

How does robots.txt control KagiBot?

A robots.txt file groups crawling rules under a User-agent line. The IETF's protocol specification defines a group as user-agent lines followed by rules and requires matching crawler names without regard to letter case.

Kagi's bot documentation gives this example, which permits crawling except for paths under /private/:

User-Agent: Kagibot
Allow: /
Disallow: /private/

The named group makes the site's KagiBot instructions explicit. In this example, Allow: / covers the site generally, while Disallow: /private/ supplies the path restriction.

The protocol requires the file to use UTF-8 encoding and the text/plain media type. It also states that robots rules are not a form of access authorization. A directory named /private/ in a robots rule therefore still needs access controls if its contents require protection.

What happens when only Googlebot has a rule?

Kagi's documented fallback applies when a robots.txt file has Googlebot directives and no KagiBot rule. Its example is:

User-Agent: *
Disallow: /
User-Agent: Googlebot
Allow: /
Disallow: /private/

Kagi explicitly says its crawler fetches pages outside /private/ with this file. The wildcard group's site-wide restriction does not prevent that documented outcome because KagiBot follows the Googlebot directives in this situation.

For a website audit, this makes the Googlebot group relevant to KagiBot access whenever a dedicated KagiBot rule is absent. Reviewing only the wildcard group misses the fallback described by Kagi.

FAQ

Can I give KagiBot its own crawling rules?

Yes. Kagi's bot documentation states that standard directives targeting Kagibot are respected. Its dedicated-group example allows the site generally and excludes /private/.

Does a wildcard block alone describe the Googlebot fallback?

Kagi's same documentation shows a wildcard block alongside a Googlebot allowance. With no KagiBot rule, KagiBot follows the Googlebot group and fetches pages outside the excluded /private/ path.

Where should I put the robots.txt file?

The IETF's access-method specification requires /robots.txt in the service's top-level path, in lowercase. The required encoding is UTF-8 and the media type is text/plain.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer