Enhanced Scraping Capacity and Reliability
This release improves the consistency of deep web crawls and expands the capabilities of our Node.js SDK. We have also refined our automated refund logic to ensure more accurate credit management for all users.
Optimized resource management now prevents performance degradation during system updates. These changes ensure your scraping requests remain uninterrupted even during peak traffic and maintenance windows.
Enhanced resource handling prevents background processes from stalling and blocking your scraping queue. This improvement leads to significantly higher success rates for complex, long-running extraction tasks.
Scraping results are now automatically managed with a 30-day retention window to keep your workspace performant. This ensures your data environment stays clean while maintaining high speeds for recent job lookups.
New Features
1Automated result retention policy
Scraping results are now automatically managed with a 30-day retention window to keep your workspace performant. This ensures your data environment stays clean while maintaining high speeds for recent job lookups.
Improvements
10Maximum Service Uptime ★
Optimized resource management now prevents performance degradation during system updates. These changes ensure your scraping requests remain uninterrupted even during peak traffic and maintenance windows.
Reliable Scrape Execution ★
Enhanced resource handling prevents background processes from stalling and blocking your scraping queue. This improvement leads to significantly higher success rates for complex, long-running extraction tasks.
Extended Job History Retention
We have updated our data policies to archive older scrape results while preserving your full job metadata indefinitely. This allows you to audit your long-term scraping history without impacting platform performance.
General bug fixes and improvements
Plus 3 internal improvements for better reliability and performance.
Optimized system-wide query performance
We have overhauled core data retrieval patterns to eliminate resource-heavy operations across our infrastructure. This results in more consistent API responsiveness and faster dashboard interactions even during global traffic peaks.
Double concurrent scraping capacity
Users can now run twice as many concurrent browser-based scraping tasks without increasing their resource footprint. This allows for significantly faster data collection at scale.
Accurate API documentation
The public API specification has been fully synchronized with the latest platform updates. Developers can rely on the documentation for the most current endpoints and schema definitions.
Faster data processing speeds
Backend data processing has been tuned to handle complex queries faster and prevent system-wide delays. This results in more consistent API response times during peak loads.
Expanded Node.js SDK Support
The Node.js SDK now supports additional parameters for scraping and extraction, including LLM provider selection and custom templates. This ensures developers have full access to our latest API features natively.
Deterministic Crawl Page Discovery
We updated our crawl engine to ensure more predictable and exhaustive results during deep site traversals. This change prevents pages from being skipped and ensures consistent data discovery regardless of crawl speed.
Bug Fixes
12Accurate Account Dashboard Metrics
Fixed a reporting discrepancy where account totals were sometimes capped by display limits. Your dashboard now reflects the true count of your team signups and usage data across all pages.
Precise Scrape Error Feedback
Improved the feedback loop for failed scraping jobs to provide more accurate error states. You will no longer see confusing "Processing" indicators or incorrect data warnings for jobs that failed before producing output.
Persistent Dashboard Preferences
Onboarding banners and UI notifications now correctly remember your dismissal preferences across different sessions. This creates a cleaner and more efficient workspace for returning developers.
General Performance and Stability Improvements
Resolved several underlying issues to prevent intermittent API client errors and improve metric calculation accuracy. These updates contribute to a more predictable and robust integration experience.
Eliminated potential request hangs
A critical update to our resource management engine prevents rare deadlocks that could occur during high-pressure scraping sessions. Your API calls will now remain stable and resolve correctly under any load condition.
Enhanced high-concurrency analytics stability
Internal data processing has been hardened to better handle simultaneous updates to your analytics data. You will see improved reliability when running multiple concurrent scraping tasks that contribute to your domain insights.
Automatic credit refund recovery
We introduced a background process that automatically identifies and refunds credits for interrupted scraping tasks. This ensures your balance is correctly restored even if a session is disconnected.
Improved URL exclusion matching
Crawl exclusion patterns are now more flexible and correctly ignore specified URLs even when leading slashes are omitted. This provides more precise control over which pages are scraped.
Enhanced dashboard error reporting
The dashboard now gracefully handles and displays page-level errors during crawls instead of crashing. This makes it easier to debug specific failures within large scraping jobs.
Reliable long-running crawls
We fixed an issue where long-running scraping tasks could prematurely expire and become unpollable. Your crawls will now remain manageable until they are fully completed.
Accurate Credit Refund Processing
We resolved an issue where automated refunds could over-calculate credits in specific crawl scenarios. Your credit balance will now more accurately reflect your actual usage and eligible refunds.
Improved Service Alert Reliability
Internal optimizations resolve intermittent conflicts during high-volume data processing and ensure system alerts are delivered reliably. This provides better visibility into job status and platform performance.
Security
1Enhanced Data Privacy
We have improved the sanitization of sensitive information in automated error reports. This ensures that personal identifiers and private keys are never exposed during troubleshooting or log analysis.
Plus 6 internal changes for stability and performance.