anti-bot

Rate Limiting

A server-side control that caps the number of requests accepted from a single IP or session within a time window, returning HTTP 429 when exceeded.

Rate limiting is a server-side mechanism that restricts how many requests a client can make within a given time window. The limit may apply per IP address, per authenticated user session, per API key, or per endpoint. Exceeding the limit returns HTTP 429 Too Many Requests, or in some cases silently degrades response quality by returning empty results or cached stale data.

Rate limit strategies vary significantly: fixed windows (10 requests per minute, window resets at :00 and :30), sliding windows (10 requests in any rolling 60-second period), and token bucket algorithms (burst allowance plus continuous refill rate). Fixed windows are susceptible to burst attacks at the boundary; sliding windows and token buckets are more resilient but also more common in modern APIs.

Effective scraping that respects rate limits uses IP rotation to distribute request volume across many origins, adds jitter between requests to avoid machine-like timing, and implements exponential backoff with jitter on 429 responses. AlterLab distributes requests across its proxy pool automatically, and the `request_interval` parameter allows setting minimum delay between requests to a single domain.

Examples

# Add delay between requests to same domain
{
  "url": "https://example.com/page/1",
  "request_interval": 2000
}

Related Terms

    Rate Limiting — Web Scraping Glossary | AlterLab