anti-bot

User-Agent Rotation

User-agent rotation changes the User-Agent header sent with each request or session to mimic different browsers, reducing the likelihood that a scraper is fingerprinted by its UA string.

The User-Agent header identifies the browser and OS making an HTTP request. Anti-bot systems flag requests with non-browser UAs (Python requests default `python-requests/2.x.x`) or with outdated browser versions. User-agent rotation cycles through a pool of realistic, up-to-date browser UA strings to avoid these signals.

Effective UA rotation goes beyond simply changing the UA string. The rest of the request headers — Accept, Accept-Language, Accept-Encoding, Sec-Fetch-* headers, and HTTP header order — must be consistent with the claimed browser. A request claiming to be Chrome 125 but sending Firefox Accept headers is internally inconsistent and detectable by fingerprinting systems.

For browser-based scrapers, the UA is set on the browser context level and automatically propagates to all headers in a coherent way. For HTTP-based scrapers, the entire header set for a given browser/OS combination should be applied as a bundle rather than swapping the UA string in isolation.

Examples

import random

UA_POOL = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36",
]

headers = {"User-Agent": random.choice(UA_POOL)}

Related Terms

User Agent
An HTTP header string identifying the browser, version, and operating system to the web server, checked by anti-bot systems against other request properties.
Browser Fingerprinting
A technique anti-bot systems use to identify bots by collecting dozens of browser attributes including Canvas, WebGL, and timing signatures.
HTTP Headers
Key-value metadata fields sent with HTTP requests and responses that communicate authentication, content type, caching, and other directives.
Stealth Mode
Browser automation patches that normalise headless browser signals to match genuine user browser profiles and navigate browser environment checks.
Anti-Bot Protection
Systems deployed by websites to detect and block automated traffic using TLS fingerprinting, CAPTCHA challenges, and behavioural analysis.

Extract User-Agent Rotation data from any website

AlterLab returns clean, structured data from any public URL — no scraper infrastructure needed. Start free, no credit card required.

View API docs
Back to Glossary

Your first scrape.
Sixty seconds.

$1 free balance. No credit card. No SDK.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · Up to 5,000 free scrapes · Balance never expires

    User-Agent Rotation — Web Scraping Glossary | AlterLab