How can a crawler rate-limit response be traced to Hostinger CDN?

Definition: A crawler rate-limit response can be traced to Hostinger CDN by checking the response status and the x-hcdn-request-id response header. Hostinger documents that a present header identifies a response that came through its CDN, while a missing header points to another layer.

Key facts

• Hostinger CDN verifies crawlers with search-engine IP lists and reverse DNS checks, and its documentation names Googlebot and Bingbot as examples. Hostinger's CDN crawler documentation describes this verification.
• Hostinger associates bursts of requests to uncached pages, including large XML sitemaps, with rate limits and 429 responses. RFC 9110 defines Retry-After as a response field for indicating how long a user agent should wait before a follow-up request.
• Google's crawler infrastructure reduces a hostname's crawl rate after significant numbers of 500, 503, or 429 responses and raises it again after those errors fall. Google's crawl-rate documentation describes both behaviors.

How does the header identify Hostinger CDN?

Hostinger's documentation, updated 15 September 2026, tells site owners to confirm that a 429 came through Hostinger CDN before changing crawler settings. The check is a live URL test followed by an inspection of the response headers. The relevant response evidence is:

429 Too Many Requests
x-hcdn-request-id: <request-id>

The page states that a missing x-hcdn-request-id means the response came from another layer, such as the website itself. That distinction matters when a reverse proxy, origin server, security service, or CDN sits between a crawler and the application. Hostinger also says that rapid requests to pages that are not cached can reach its protective rate limits and return 429.

Which crawler-rate settings can change?

The Hostinger documentation separates CDN diagnosis from crawler controls. It says that persistent Bing 429 errors can be addressed by lowering the crawl rate in Bing Webmaster Tools under Configure My Site, then Crawl Control. It also says that Googlebot adjusts its crawl rate automatically. Hostinger's own dashboard cannot change crawler rate limits, so a persistent Hostinger-side case requires the response evidence and a review by Hostinger support.

The same page says that occasional 429 responses during request bursts are normal and that crawlers retry automatically. A single response therefore records a rate-limit event, while repeated failures require a check of the response path and the crawler-specific control documented by the relevant search engine.

What does Google document about rate reduction?

Google's crawl-rate documentation, updated 6 October 2026, says its crawler infrastructure uses algorithms to determine a site's optimal crawl rate. For a short emergency period, it documents returning 500, 503, or 429 to crawl requests instead of 200. Google says a significant number of these responses reduces the crawl rate across the whole hostname, including URLs that return errors and URLs that return content. It automatically raises the rate after the number of errors is reduced.

Google also documents a special request for an unusually high crawl rate when serving errors is not feasible. The page says that a site owner cannot request an increase in crawl rate and that evaluation of the special request may take several days. The documented emergency method therefore has a broad hostname scope and a short intended duration.

How should the response be traced?

Start with a live URL test for the affected page. Record the URL, crawler, time, status code, and all response headers. A 429 with x-hcdn-request-id supports the diagnosis that Hostinger CDN produced the rate-limit response. A 429 without that header sends the investigation to another layer. Hostinger's troubleshooting steps also name Google Search Console URL Inspection, Google Search Console's Settings, Crawl stats, and Bing Webmaster Tools URL Inspection as checks after a settings change.

For retry timing, RFC 9110 says that servers send Retry-After to indicate how long a user agent ought to wait before a follow-up request. Its value can be an HTTP date or a number of seconds. That field is separate from x-hcdn-request-id: one records the response path, while the other can state a retry interval.

Related terms

• HTTP 429 Too Many Requests
• Retry-After
• Server log analysis
• Bot crawl frequency
• Bot response code distribution

FAQ

Does x-hcdn-request-id prove that a crawler is genuine?

No. The header traces the response path through Hostinger CDN. Hostinger documents crawler authenticity separately through search-engine IP address lists and reverse DNS checks.

Can you change Hostinger's crawler rate limit in the dashboard?

No. Hostinger documents that crawler rate limits cannot be changed from its dashboard. A persistent case should include the status code, URL, time, and available x-hcdn-request-id value for support review.

Can you lower Google's crawl rate from Hostinger?

No. Hostinger says Googlebot adjusts its crawl rate automatically. Google documents temporary error responses and a special request for an unusually high crawl rate as separate controls.

Sources

1. Hostinger, "Hostinger CDN: Search engine crawlers and SEO", 15 September 2026
2. Google for Developers, "Reduce the Google crawl rate", 6 October 2026
3. IETF, "RFC 9110: HTTP Semantics", June 2022

---

Check how a page reads to answer engines. Open the GEO score analyzer