In short. Use HTTP 429 for excessive requests and provide Retry-After when appropriate, keeping Google overload responses temporary.
Definition: HTTP 429 Too Many Requests is the response status for a client exceeding a request rate limit. A server can include a Retry-After header to indicate how long the client should wait before making another request.
Key facts
- RFC 6585, section 4 defines 429 and recommends that the response explain the rate-limit condition.
- RFC 9110, section 10.2.3 defines Retry-After as either a date and time or a delay in seconds.
- As of October 2026, Google's crawl-rate guidance documents hostname-wide slowing when a significant number of URLs return 500, 503 or 429.
- Google's HTTP status documentation treats 429 as a server-overload signal and advises against using 401 or 403 to limit crawling.
Which status should the server use?
RFC 6585 assigns 429 to excessive requests within a given period. The server chooses how to count requests and identify the client. The specification allows counting per resource, across a server or across several servers. Responses with this status must not be stored by a cache.
HTTP 503 Service Unavailable describes temporary overload or scheduled maintenance under RFC 9110, section 15.6.4. A server can therefore use 429 when a client exceeds its request allowance, or 503 when temporary service unavailability describes the condition. Both permit a Retry-After signal.
What retry information belongs in the response?
The header portion of the example in RFC 6585 is:
HTTP/1.1 429 Too Many Requests
Content-Type: text/html
Retry-After: 3600
Here, the waiting period is 3600 seconds. This is the RFC's example value; the response should explain the actual rate-limit condition in its body.
RFC 9110 specifies that a delay is a non-negative decimal integer measured from receipt of the response. Its alternative form is an HTTP date. Google's crawl-rate guidance explicitly allows either form with 503 or 429 to indicate when its crawlers can retry.
What happens to Google's crawling and indexing?
Google's emergency crawl-rate instructions describe temporary error responses as a short-term measure. When enough URLs return those errors, crawling slows across the whole hostname, including URLs that still return content. The rate automatically increases again as errors decrease.
The same guidance warns against continuing this for longer than one or two days: repeated errors on a URL over multiple days can lead to its removal from Google's index. Slower crawling also delays refreshing existing pages, including changes to prices and availability.
For diagnosis, Google recommends reviewing recent server access logs with the hosting company. Its listed causes of sharp increases include faceted navigation, which creates URLs for filters and sorting, and calendars with many date-specific URLs. Addressing such a cause belongs alongside the temporary response change.
Related terms
FAQ
Can I use 403 to slow Googlebot?
Google's status-code documentation advises against 401 and 403 for crawl-rate limiting. It identifies 429 as the exception among client-error statuses that signals overload.
Is Retry-After mandatory with 429?
RFC 6585 makes the header optional. It recommends including details that explain the condition in the response body.
Must a server answer every excessive request with 429?
RFC 6585, section 7.2 recognizes that producing error responses consumes resources. Under attack or very heavy request volume, a server may drop connections or take other steps to limit resource use.
Sources
- IETF, "Additional HTTP Status Codes", April 2012
- IETF, "HTTP Semantics", June 2022
- Google, "Reduce the Google crawl rate", 6 October 2026
- Google, "How HTTP status codes affect Google's crawlers", 4 February 2026
Check how a page reads to answer engines. Open the GEO score analyzer