protocol

HTTP Headers

Key-value metadata fields sent with HTTP requests and responses that communicate authentication, content type, caching, and other directives.

HTTP headers are key-value pairs transmitted at the start of every HTTP request and response that carry metadata about the message. Request headers tell the server about the client (User-Agent, Accept-Language, Accept-Encoding), the content being sent (Content-Type, Content-Length), authentication credentials (Authorization, X-API-Key), and caching preferences (If-None-Match, Cache-Control). Response headers tell the client about the server's response and how to handle it.

In web scraping, headers are a significant detection surface. Anti-bot systems validate that request headers match what a real browser would send for that specific browser version and operating system. A mismatch between the User-Agent claiming to be Chrome 124 on Windows and a missing `sec-ch-ua` client hint header, or a non-browser `Accept-Encoding` value, is a strong bot signal.

Critical headers to get right for anti-bot evasion: `User-Agent`, `Accept`, `Accept-Encoding`, `Accept-Language`, `sec-ch-ua`, `sec-ch-ua-mobile`, `sec-ch-ua-platform`, `Sec-Fetch-Site`, `Sec-Fetch-Mode`, `Sec-Fetch-Dest`. AlterLab manages the full header set automatically, matching the exact header profile of the Chromium version in use.

Examples

# Headers a real Chrome browser sends
GET /page HTTP/2
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36...
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
Accept-Encoding: gzip, deflate, br
sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124"
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate

Related Terms

User Agent
An HTTP header string identifying the browser, version, and operating system to the web server, checked by anti-bot systems against other request properties.
TLS Fingerprint
A signature derived from TLS handshake parameters a client sends when opening an HTTPS connection, used by anti-bot systems to classify traffic.
Anti-Bot Protection
Systems deployed by websites to detect and block automated traffic using TLS fingerprinting, CAPTCHA challenges, and behavioural analysis.
REST API
A web service that exposes data and actions through standard HTTP methods and resource-oriented URLs, returning structured JSON responses.
Back to Glossary
Read the DocsTry the API
    HTTP Headers — Web Scraping Glossary | AlterLab