Web Scraping Success Rate and Cost by Defence Class: October 2026 Data

Most scraping benchmarks are run on a handful of hand-picked pages. This one is not. It is an aggregate of 90,251 customer scrape jobs that ran on AlterLab in the 30 days ending 30 October 2026 (window 2026-09-30 to 2026-10-30, end exclusive), grouped by the kind of defence each page served and the fetch tier that finished the job.

Across all jobs the success rate was 75.9%. Across the 18,471 jobs on pages that served a defence it was 41.6%. That gap is the work in front of us, and the tables below show where it sits.

Success rate by defence class

Each bar is the share of jobs in that class that returned the requested content. Classes with fewer than 100 jobs are left out of the dataset, so a small class never exposes a handful of requests.

Two things stand out. Pages with no detected defence still fail about 15% of the time, mostly on tier 3 and tier 4 jobs that were escalated because the first attempts did not return content. And the two lowest classes, DataDome at 3.8% and Imperva at 9.8%, are not a limit of the web. They are open engineering problems on our side, and each low cell is tracked as a regression to investigate and fix at the class level.

Success rate by fetch tier

The tier is the rung of the fetch ladder that finished the job. Jobs are routed to the lowest tier that works, so higher tiers receive the harder pages and their success rates are not comparable across tiers on a like-for-like basis.

Tier 1 and tier 2 jobs on pages with no detected defence succeed 98.6% and 97.8% of the time. Pages that serve a defence rarely finish below tier 3, which is why the cells for those classes start there. DataDome jobs that finish on tier 3 returned content 0.0% of the time in this window, while tier 4 returned it 33.5% of the time, so the tier 3 route for that class is the first thing we are fixing.

Median billed cost

Cost is the price billed for a successful job, not infrastructure cost. The median is exact, per cell.

A page with no detected defence costs $0.30 per 1,000 successful requests at the median. Pages that need tier 3 or tier 4 cost $2.00 to $4.00. That is the reason to solve at the lowest tier that works: every job that finishes a rung lower is both cheaper and more likely to succeed.

Method

• Source: AlterLab production job telemetry, read with the committed read-only query infra/reports/antibot_benchmark_monthly.sql. Anyone with the schema can rerun it and get the same numbers.
• Window: 30 days, 2026-09-30 to 2026-10-30 (end exclusive).
• Cohort: customer jobs only. Internal, admin, test and first-party crawler accounts are excluded.
• Class: the defence detected on the page for each request. "Other defences" groups defences outside the named families.
• Tiers: post-hoc tiers 3.5 and 4.5 count toward the rung that fetched (3 and 4). Cache replays are excluded.
• Privacy: no customer, domain, URL or job identifiers appear in the data, and cells under 100 jobs are omitted.
• Not a guarantee: these are measured results for one window, not a promise for any page. Success depends on the page, the request and the day.

Download the data

The full dataset is open under CC-BY-4.0:

• Download the CSV (one row per defence class and tier)
• Dataset page with schema and history

It is refreshed monthly. If you cite it, link to the dataset page.

Try AlterLab free with up to 5,000 free requests, and see the tier your own pages finish on.