Web Scraping API Benchmarks
Original performance data from AlterLab's 5-tier scraping infrastructure — success rates by website category, geographic latency, success rates by protection type, and cost comparisons. Updated June 2026.
Anti-Bot Difficulty Index
A neutral, dated, vendor-level read on how hard each anti-bot system is to bypass right now — and whether it is trending harder or easier. Computed continuously from real production scrape outcomes, not a lab benchmark.
14,411 verified scrape encounters as of August 3, 2026, 90-day window. Full methodology.
| Vendor | Difficulty Score | Trend | Sample Size |
|---|---|---|---|
| datadome | 99.7 | 0.0 | 3,265 |
| small_block_page | 97.4 | 0.0 | 76 |
| akamai_waf | 97.1 | 0.0 | 34 |
| aws_waf | 95.6 | 0.0 | 1,259 |
| imperva_soft | 89.0 | 0.0 | 254 |
| access_denied | 75.0 | 0.0 | 24 |
| cloudflare_managed_challenge | 70.4 | 0.0 | 54 |
| soft_block | 69.9 | 0.0 | 561 |
| cloudflare_turnstile | 64.1 | 0.0 | 217 |
| perimeterx_hold_button | 61.8 | 0.0 | 34 |
| cloudflare_js | 56.0 | 0.0 | 1,394 |
| imperva_js | 38.1 | 0.0 | 21 |
| captcha | 23.1 | 0.0 | 2,048 |
| cloudflare_blocked | 15.5 | 0.0 | 1,044 |
| js_render_required | 9.3 | 0.0 | 54 |
| perimeterx | 6.8 | 0.0 | 369 |
| blocked | 5.1 | 0.0 | 1,049 |
| cloudflare | 0.0 | 0.0 | 2,654 |
Vendor-level aggregate only — no per-domain data, no cost figures, no raw bypass-success percentage. Embed this data (CC BY 4.0) via the embeddable widget.
Anti-Bot Success Rates by Tier
Each tier adds capability and cost. AlterLab auto-escalates from the cheapest tier, so you only pay for what's needed.
Success Rates by Website Category
Aggregated success rates across 6 major website categories. These are AlterLab's own operational estimates, not third-party numbers — see the live, independently verifiable Anti-Bot Difficulty Index above for the dataset backed by real-time production telemetry.
Methodology: success rate = requests returning valid HTML content / total requests. Data from January–June 2026 production traffic.
Response Latency by Geographic Region
End-to-end latency from API request to response delivery, measured across AlterLab's proxy node network. Proxy nodes are co-located in each region to minimize hops.
Methodology: p50 = median response time, p95 = 95th percentile. Measured at Tier 1 (Curl) baseline. Higher tiers add rendering overhead. Data from Q1–Q2 2026.
Success Rates by Protection Type
How AlterLab performs against each class of website protection mechanism, measured against real production targets. Success rate is the percentage of requests that return valid content when the protection type is present.
Methodology: protection type classified by AlterLab's detection layer before tier selection. Success measured at the first tier that handles the detected protection class. Data from Q1–Q2 2026 production traffic.
Success Rates by Domain
Per-domain performance across 20 frequently scraped sites. Success rate, median latency, cost per successful request, and the recommended tier for each target.
Methodology: success rate = requests returning valid HTML / total requests per domain. Median latency is end-to-end including proxy routing. Cost per success reflects the tier that typically resolves the domain. Data from Q1–Q2 2026 production traffic. Domains with fewer than 50K requests excluded.
| Domain | Category | Success Rate | Median Latency | Cost / Success | Rec. Tier |
|---|---|---|---|---|---|
| amazon.com | E-commerce | 99% | 980ms | $0.0003 | T2 |
| reddit.com | Social / Community | 98% | 620ms | $0.0002 | T1–T2 |
| linkedin.com | Professional Network | 91% | 4,100ms | $0.004 | T4 |
| zillow.com | Real Estate | 95% | 2,800ms | $0.002 | T3 |
| indeed.com | Job Board | 93% | 3,200ms | $0.002 | T3 |
| glassdoor.com | Job Board | 90% | 5,200ms | $0.004 | T4 |
| yelp.com | Reviews / Directories | 94% | 2,400ms | $0.002 | T3 |
| airbnb.com | Travel / Marketplace | 92% | 4,800ms | $0.004 | T4 |
| booking.com | Travel / Marketplace | 96% | 1,900ms | $0.002 | T3 |
| walmart.com | E-commerce | 97% | 1,400ms | $0.0003 | T2 |
| etsy.com | E-commerce | 99% | 850ms | $0.0002 | T1–T2 |
| realtor.com | Real Estate | 96% | 2,100ms | $0.002 | T2–T3 |
| bloomberg.com | Financial News | 88% | 6,400ms | $0.004 | T4 |
| reuters.com | News & Media | 99% | 590ms | $0.0002 | T1 |
| sec.gov | Government / Financial | 98% | 1,100ms | $0.0002 | T1 |
| trulia.com | Real Estate | 95% | 2,600ms | $0.002 | T3 |
| expedia.com | Travel / Marketplace | 97% | 1,700ms | $0.0003 | T2 |
| yellowpages.com | Business Directory | 98% | 780ms | $0.0002 | T1 |
| crunchbase.com | Business Intelligence | 89% | 5,800ms | $0.004 | T4 |
| g2.com | Reviews / Software | 93% | 3,400ms | $0.002 | T3 |
Response Time Benchmarks
End-to-end response times including proxy routing, rendering, and content extraction. Measured from API request to response delivery.
| Tier | Method | Typical Range | Visualization |
|---|---|---|---|
| T1: Curl | Ultra-fast curl binary | 200–500ms | |
| T2: HTTP | HTTPX with TLS fingerprinting & HTTP/2 | 500–2,000ms | |
| T3: Stealth | curl_cffi with Chrome browser impersonation | 2–5s | |
| T4: Browser | Playwright browser automation | 5–15s | |
| T5: Captcha | Browser with AI-powered challenge resolution | 10–30s |
Cost Comparison
Per-request costs across major scraping API providers. AlterLab's pay-per-request model has no monthly minimums or expiring balance.
| Provider | Basic Scrape | JS Rendering | Captcha Solve | Pricing Model |
|---|---|---|---|---|
| AlterLabYou are here | $0.0002 | $0.004 | $0.02 | Pay-per-request |
| ScraperAPI | $0.001 | $0.005 | $0.015 | Monthly subscription |
| Bright Data | $0.001 | $0.01 | $0.03 | Monthly + bandwidth |
| Firecrawl | $0.002 | $0.002 | N/A | Monthly subscription |
| ScrapingBee | $0.001 | $0.005 | $0.015 | Monthly subscription |
Competitor pricing based on publicly available pricing pages as of June 2026. Actual costs may vary by plan and volume.
Anti-Bot System Coverage
AlterLab's multi-tier architecture handles every major anti-bot system in production today.
How Auto-Escalation Works
AlterLab automatically routes each request through the optimal tier, minimizing cost while maximizing success rate.
Start Cheap
Every request begins at Tier 1 ($0.0002). If the target responds with valid HTML, you're done.
Detect & Escalate
If blocked, AlterLab detects the anti-bot system and jumps to the right tier — no wasted intermediate attempts.
Pay for Success
You're charged only for the tier that succeeds. Domain intelligence learns optimal tiers for future requests.
Methodology
For the live Anti-Bot Difficulty Index above, see the dedicated, dated Difficulty Index methodology page for the exact scoring formula, trend computation, and minimum sample floor. The sections below describe the separate historical benchmark dataset.
All benchmark data is sourced from aggregated production traffic on AlterLab's scraping infrastructure. No synthetic benchmarks or controlled lab environments — these are real request outcomes from paying customers, anonymized and aggregated by category, region, and protection type.
Success Rate Definition
Success rate = requests returning valid HTML content / total requests attempted at a given tier or against a given protection type. The “overall 98%+ success rate” reflects AlterLab's auto-escalation system: the system routes requests through progressively more capable tiers until a valid response is obtained.
Website Category Classification
Categories are assigned using AlterLab's domain intelligence layer based on domain metadata and content type signals. Sample sizes reflect the number of requests classified in each category from January–June 2026 production traffic. Categories with fewer than 100K requests are excluded to ensure statistical confidence.
Geographic Latency
Latency is measured end-to-end from API request receipt to response delivery, including proxy routing, DNS resolution, TLS handshake, page rendering (where applicable), and content extraction. p50 is the median response time; p95 is the 95th percentile. Baseline measurements use Tier 1 (Curl) — higher tiers add rendering overhead. Proxy nodes are deployed in each reported region.
Protection Type Classification
AlterLab's detection layer classifies the primary protection mechanism encountered during each request. When multiple protection types are present, the most restrictive is used for classification. Success rate for a protection type reflects performance specifically against that class of challenge at the appropriate handling tier.
Competitor Pricing
Competitor pricing data is sourced from publicly available pricing pages as of June 2026 and may not reflect negotiated enterprise rates or volume discounts.
More Web Scraping API Resources
Part of the Web Scraping API Documentation cluster
Complete API reference with 5-tier auto-escalation — Curl to challenge resolution.
Pillar pageSide-by-side comparison with ScraperAPI, Firecrawl, Bright Data, and Apify.
Use web scraping in Claude, GPT-4, and LangChain agent workflows.
Handles bot-protected websites, JavaScript challenges, and complex protections automatically.
Full browser rendering for SPAs, React, and dynamic content.
Chain Search → Crawl → Map operations in a single API workflow.
From $0.0002/request. Balance never expires — no monthly subscription.
Test your own URL
See how AlterLab performs against any URL you care about — no account required. The playground runs a real scrape and returns timing, tier used, and the extracted content.
Open playgroundYour first scrape.
Sixty seconds.
$1 free credit — up to 5,000 scrapes. No credit card.
Just a POST request.
No credit card required · $1 free credit, up to 5,000 scrapes · Balance never expires