Can an explicit Hostinger CDN bot rule block a verified search crawler?

Definition: Yes. Hostinger CDN can return 403 to a bot that has passed its crawler verification when that bot is explicitly blocked in the CDN settings. Verification identifies the crawler, while the explicit bot rule determines access.

Key facts

• Hostinger says its CDN checks crawler IP address lists published by search engines and reverse DNS. Hostinger's CDN crawler documentation describes both checks.
• Requests that only pretend to be Googlebot or Bingbot fail Hostinger's verification.
• A bot that is explicitly blocked in Hostinger CDN settings receives 403, even when Hostinger has verified it.
• Hostinger's Under Attack mode challenges every visitor with a browser verification page, and search engine crawlers do not execute that verification.
• Google says common crawlers such as Googlebot always respect robots.txt rules for automatic crawls. Google's crawler verification documentation describes this crawler category.
• RFC 9309 says robots rules control crawler access to URIs and are not a form of access authorization. The IETF Robots Exclusion Protocol defines the protocol.

What does Hostinger verify before allowing a crawler?

Hostinger's CDN verifies crawlers against official IP address lists published by search engines and uses reverse DNS checks. A user agent string alone therefore does not establish that a request comes from Googlebot or Bingbot. Hostinger says that requests pretending to be either crawler fail verification.

Google documents a matching procedure for Google requests. Run a reverse DNS lookup on the IP address in the server logs, check that the returned name ends in googlebot.com, google.com, or googleusercontent.com, then run a forward lookup and confirm that it returns the original IP address.

host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.

host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

This test concerns crawler identity. It does not remove a separate Hostinger CDN bot block.

Does an explicit bot block override verification?

Yes. Hostinger states that bots blocked explicitly in CDN settings receive a 403 response even when they are verified. The documented result makes the explicit block decisive for the request. The same page lists Under Attack mode, country and IP blocking rules, bot blocking rules, and crawl rate as separate causes of crawler errors.

robots.txt operates at a different layer. RFC 9309 describes rules that crawlers are requested to honor when accessing URIs and states that these rules are not access authorization. Google also documents that common crawlers respect robots.txt during automatic crawls. A permissive robots rule therefore cannot cancel a Hostinger CDN access decision that returns 403.

How can you check which Hostinger setting is responsible?

Test the affected URL after each change. Hostinger directs site owners to use Google Search Console's URL Inspection live test, review Crawl stats, or run a live test in Bing Webmaster Tools. A successful live test confirms that the crawler reaches the site through Hostinger CDN.

Separate a block from a rate limit by checking the response. Hostinger associates an explicit bot block with 403. It says that rapid bursts against uncached pages such as large XML sitemaps can return 429, and that crawlers retry after occasional burst responses. For a 429, Hostinger directs you to check for its x-hcdn-request-id response header before changing crawler settings. If a verified crawler still receives 403 or 429 after the documented settings are reviewed, Hostinger directs the owner to contact support with the exact URL, status code, failed-crawl time, and available request ID.

Related terms

• Googlebot
• Bingbot
• robots.txt User-agent directive
• robots.txt group precedence

FAQ

Can Hostinger verification allow a bot that is explicitly blocked?

No. Hostinger documents a 403 response for an explicitly blocked bot, including a bot that has been verified.

How can you check whether a request is really from Googlebot?

Use Google's reverse DNS and forward DNS procedure. The reverse name must use a Google-owned domain, and the forward lookup must return the original IP address.

Can a robots.txt rule override the Hostinger CDN block?

No. RFC 9309 treats robots rules as crawler instructions, while Hostinger's explicit bot rule controls the CDN response.

What does a 429 suggest in this setup?

Hostinger associates 429 with rate limits reached by rapid bursts of requests to uncached pages. Its documentation says crawlers retry after occasional burst responses.

Sources

1. Hostinger, "Hostinger CDN: Search engine crawlers and SEO", 15 September 2026
2. Google Search Central, "Verify requests from Google crawlers and fetchers", 20 March 2026
3. IETF, "Robots Exclusion Protocol", September 2022

---

Check how a page reads to answer engines. Open the GEO score analyzer