In short. HTTP 429 signals rate limiting, can include Retry-After, and causes Google’s crawlers to slow down temporarily.

Definition: HTTP 429 Too Many Requests is the response status defined by RFC 6585 for a user sending too many requests within a given time. A server can include Retry-After to indicate a wait before another request.

Key facts

RFC 6585 says the response should explain the condition and must stay out of caches.
RFC 9110 defines Retry-After as either an HTTP date or a delay in seconds.
As of 17 October 2026, Google's crawler documentation treats 429 as a server overload signal that temporarily slows crawling.

What determines the rate limit?

A rate limit controls request frequency over time. RFC 6585, section 4, allows counting per resource, across a server, or across several servers. Authentication credentials or a stateful cookie can identify the user whose requests count toward the limit.

The RFC gives an illustrative policy of 50 requests per hour per logged-in user. That quota belongs to its example; the specification leaves identification and counting policy to the server operator. A response body should explain the condition so the recipient can understand the applicable restriction.

How does Retry-After guide a retry?

The following response headers reproduce the example in RFC 6585:

HTTP/1.1 429 Too Many Requests
Content-Type: text/html
Retry-After: 3600

RFC 9110, section 10.2.3, explains that a numeric value measures seconds after receipt of the response. Here, the indicated wait is 3600 seconds. An HTTP date instead identifies a point in time. Numeric delays use non-negative decimal integers.

Retry timing also requires attention to the operation being repeated. The same RFC defines an idempotent method as one whose repeated identical requests have the same intended server effect as a single request. It advises clients to retry non-idempotent requests automatically only when they know the operation is effectively idempotent or can detect that the original request was never applied.

What does this change for crawling?

Google's HTTP status guidance groups 429 with server errors for crawler handling. For Google Search, already indexed URLs initially remain in the index but are eventually dropped if these errors persist. When successful responses resume, Google gradually increases the site's crawl rate.

This makes persistent rate limiting relevant to a website's availability to Google's crawlers. The documented recovery signal is a return to successful HTTP responses. These handling rules describe Google's infrastructure specifically.

FAQ

Must every 429 response include Retry-After?

RFC 6585 makes that header optional. It separately recommends an explanation of the rate-limit condition in the response body.

How can you interpret a numeric Retry-After value?

Under RFC 9110, you count the indicated seconds from receipt of the response. Its example Retry-After: 120 means a delay of two minutes.

Should you use 403 to slow Google's crawlers?

Google's guidance explicitly advises against using 401 or 403 for crawl-rate limiting. It identifies 429 as the exception among client-error responses that affects crawl rate.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer