A TLS (Transport Layer Security) certificate is a digital document issued by a Certificate Authority (CA) that cryptographically proves a server's identity. When a scraper connects to `https://example.com`, the server presents its certificate; the client verifies that it is signed by a trusted CA, matches the domain name, and has not expired before proceeding with the encrypted connection.
For scrapers, TLS certificate validation is normally transparent — the HTTP client library handles it automatically. However, in testing and debugging contexts, developers sometimes disable certificate validation (`verify=False` in Python requests) to capture traffic through an intercepting proxy like Charles or mitmproxy. This creates a security risk and should never be done in production.
Anti-bot systems may inspect the TLS certificate presented during HTTPS connections to intermediate proxies to detect interception. Mutual TLS (mTLS) is used by some enterprise APIs to require client certificates in addition to server certificates, adding an authentication layer that scrapers must satisfy.