Enhanced Concurrency and Payment Protection
This release includes 1 update.
The billing deposit process is now fully resilient, preserving your payment details if a connection error occurs. Retries are idempotent, ensuring you are never double-charged.
Standard workers have been upgraded to support sandboxed virtual displays for interactive verification challenges. This ensures higher success rates when scraping heavily protected websites.
Scrape classifications are now aligned with billing rules to ensure that requests returning error pages are handled as failures. This avoids confusing charges and saves you money.
New Features
1Resilient Billing Checkout Recovery
The billing deposit process is now fully resilient, preserving your payment details if a connection error occurs. Retries are idempotent, ensuring you are never double-charged.
Improvements
16Reliable Website Challenge Solving
Standard workers have been upgraded to support sandboxed virtual displays for interactive verification challenges. This ensures higher success rates when scraping heavily protected websites.
Accurate Billing for Scrape Errors
Scrape classifications are now aligned with billing rules to ensure that requests returning error pages are handled as failures. This avoids confusing charges and saves you money.
Faster Job Status Failure Feedback
Status checks for jobs, batches, and crawls now fail immediately on non-retryable errors rather than waiting for the timeout. This speeds up your API integration feedback loops.
Richer Diagnostic Evidence for Retries
Diagnostic snapshots for retried scraping jobs now preserve the most complete terminal evidence instead of being overwritten by earlier attempts. This provides highly detailed info for debugging.
Optimized HTML Parsing Latency
We optimized the HTML parsing engine to process large or deeply nested markup elements in linear time. This dramatically reduces parsing overhead and eliminates processing hangs on complex pages.
Accurate Page Status Detection
Content validation algorithms now ignore hidden marker phrases nested within scripts or comments when evaluating page structures. This guarantees your status checks only match visible text.
General bug fixes and improvements
Plus 11 internal improvements for better reliability and performance.
Immediate capacity recovery
Your account and domain concurrency limits are now immediately freed up the moment a job finishes, rather than waiting for a rolling window to expire. This lets you run sequential scraping batches much faster without hitting artificial capacity caps.
Smarter challenge bypass
The scraper is now smart enough to ignore inert or hidden anti-bot challenge placeholders on genuine pages. This prevents false alarms and ensures your requests load successfully without being blocked by inactive detection systems.
Robust crawl scheduling
Crawling and scheduling routines are now significantly more resilient against transient connection or authorization issues. Instead of failing immediately, the system tolerates short-lived backend hiccups before declaring a crawl failed.
Accurate performance metrics
Scraping analytics and aggregate success metrics now automatically exclude incomplete or legacy diagnostic records. This ensures that your dashboard historical success rates and anti-bot census data are cleaner and more accurate.
Robust proxy rotation for BYOP
Custom proxy configurations now strictly enforce dynamic IP rotation capabilities. This ensures static or unmanaged proxy credentials no longer cause exit node duplication or connection failures during scraping.
Adaptive proxy tiering
Headless requests now feed validation failures directly into our routing engine. If a proxy path is rejected, the system dynamically swaps routes to maximize your request success rates.
Guaranteed credit refunds
We have redesigned our transaction rollback flow to guarantee partial credit refunds. If a crawl is cancelled or a batch search fails mid-loop, your credits are safely and instantly returned to your balance.
Predictable rendering timeouts
Advanced rendering tasks now enforce a strict, unified deadline across all internal retries. Headless browser capacity limits are handled cleanly, allowing requests to fail fast rather than hanging indefinitely.
fix(backup): verify local backups against their published contract version
Fixes , the deploy blocker that failed production deploy run 37634854383 at.
Bug Fixes
15Improved Organization Billing Reliability
Organization billing and usage statistics endpoints no longer crash when an active owner is not assigned. Additionally, sensitive owner details are safely restricted on standard endpoints.
Correct Default Checkout Payment Mode
One-time checkout deposits now default to the correct payment-only mode rather than erroneously trying to establish recurring subscriptions. This ensures that account funding runs smoothly.
Smoother Checkout Form Recovery State
The checkout interface now automatically resets and unlocks on payload or session errors during deposit preparation. This prevents payment screens from getting stuck and allows quick retries.
Sanitized Crawl Error Export Messages
Error details in failed or rejected crawl status pages and export reports are now properly sanitized. This prevents internal scheduling details from being exposed in public exports.
No duplicate checkouts
When completing a deposit checkout, the billing form now automatically resumes your existing payment session in the event of a conflict. This avoids forcing you to start a new session, preventing accidental duplicate charges.
Guaranteed deposit bonuses
Delayed payment completions that settle after typical promotional periods are now automatically recovered and credited. You will no longer lose your promotional bonus if a payment takes longer than usual to process.
Fail-safe organization billing
If the organization billing system encounters a temporary database error, requests will now cleanly fail closed with a 503 status instead of accidentally falling back to individual team members' wallets. This protects personal team accounts from unexpected charges.
Stable deep crawls
Crawls with large discovery and URL expansion phases will no longer time out or fail with closed connection errors. Balance verification is now decoupled from URL discovery to ensure stable execution.
Reliable limit enforcement
The system now eliminates race conditions where concurrent API requests could temporarily leak and bypass your domain dispatch limits. This prevents unexpected rate limiting or concurrent usage spikes during high-volume periods.
Consistent scraping diagnostics
Diagnostic snapshot records now accurately prioritize your final successful scrape attempt over previous failed retries. This ensures that your diagnostic dashboard logs always reflect correct execution states.
Resilient diagnostics API
The diagnostics API is now highly tolerant of malformed attempt metadata, dropping legacy invalid elements rather than raising 500 errors. This ensures your integration can always access diagnostics history without interruptions.
Accurate streaming challenge detection
Our HTML stream parser has been optimized to prevent false-positive challenge detection on inert page templates. This ensures your scraping jobs do not prematurely abort active streaming responses.
Idempotent search child debits
We resolved a potential double-billing bug by making child search enqueuing fully idempotent. Retrying a parent search job will no longer duplicate debits or enqueue duplicate child tasks.
Consistent rendering diagnostics
Failed headless browser rendering tasks now carry a uniform, versioned diagnostic payload. This ensures you always receive structured, readable evidence files regardless of why a task failed.
Reliable connection racing
Concurrent connection racing has been refined to fully validate streaming response content before choosing a winner. This avoids premature cancellation of active connections during verification screens.
Security
7Isolated Proxy Session Pooling
Browser session pooling now isolates contexts strictly by their exact proxy URL and owner credentials. This prevents potential session leakage when sharing gateway hosts across scraper instances.
Enhanced API Key Redirection Security
By disabling automatic redirect following in development verification checks, your sensitive API keys are prevented from being sent to external redirect targets. This enforces a secure boundary.
Hardened billing sessions
Legacy, unvalidated payment checkout paths have been retired to enhance platform security. Checkout endpoints now strictly fail closed if payment sessions do not provide explicit metadata validation.
Secure deposit processing
Deposits flagged for potential fraud are now completely terminated immediately upon refund. This ensures that associated promotional bonuses or automatic recharges do not execute for cancelled sessions.
Strict API key enforcement
Deactivated or deleted users are now immediately rejected at the API authorization boundary. Service-account and personal keys will no longer permit access once a user is soft-deleted.
Secure billing boundaries during transfer
Guards during organization ownership transfers prevent in-flight charges from being billed to the prior owner. Debits and ownership changes are now strictly isolated.
Fail-closed billing resolver
Organizations lacking active owners or encountering undefined billing states now safely fail closed. This prevents accidental billing exposure and unauthorized credit consumption.
Plus 33 internal changes for stability and performance.