Enhanced Data Exports and Live Benchmarks
This release introduces complete, full-size data exports in CSV and JSONL formats alongside a brand-new live difficulty benchmark dashboard. We have also launched official AI framework SDKs and resolved several dashboard reporting discrepancies to elevate your developer experience.
You can now stream complete bulk datasets in JSONL or CSV formats directly from your dashboard or via the API. This bypasses previous limits, providing complete, untruncated results in a single click.
Compare scraping performance across major detection systems using our new public difficulty benchmarks. You can fetch live index scores programmatically or display them with our embeddable widget.
Analyze web environments before starting a crawl with our new open-source MIT-licensed package and command-line utility. This tool cleanly identifies which system protects any given URL.
New Features
4Untruncated Bulk Data Downloads
You can now stream complete bulk datasets in JSONL or CSV formats directly from your dashboard or via the API. This bypasses previous limits, providing complete, untruncated results in a single click.
Public Detection Difficulty Benchmarks
Compare scraping performance across major detection systems using our new public difficulty benchmarks. You can fetch live index scores programmatically or display them with our embeddable widget.
Open-Source Detection Package
Analyze web environments before starting a crawl with our new open-source MIT-licensed package and command-line utility. This tool cleanly identifies which system protects any given URL.
Real-Time Enterprise Usage Analytics
Track billing quotas, ad metrics, and job completion statistics in real time. These insights are now available programmatically and on a dedicated dashboard usage card.
Improvements
5High-Performance Large Datasets Pagination
Exporting large datasets is now significantly faster. We optimized our query sorting and streaming infrastructure to prevent performance degradation when traversing massive jobs.
Optimized Media and Asset Crawling
Our scrapers now dynamically profile and mimic asset sub-resource requests. This ensures higher reliability when downloading stylesheets, fonts, and images from CDNs.
Seamless New Tenant Provisioning
Changes to dedicated crawl environments and enterprise configs now activate instantly. This eliminates the need to coordinate manual crawler restarts or wait for redeployments.
Auto-Configured Onboarding API Keys
Enterprise-level API keys created during setup now automatically receive the correct dashboard permissions. This eliminates manual configuration and prevents credential mismatches.
General bug fixes and improvements
Plus 31 internal improvements for better reliability and performance.
Bug Fixes
3Restored Complete Metrics Cycles
Resolved a timezone bug where the first calendar day of usage data was omitted from dashboard charts. Your usage and billing history now displays fully complete periods.
Streamlined Dashboard Download Interactions
Polished the visual styling of download buttons on the bulk results page. This improves layout consistency and provides clearer feedback during large exports.
Onboarding Documentation Path Restored
Corrected a broken link on the final step of the setup sequence. New users are now seamlessly routed to the correct enterprise integration documentation page.
Security
1Protected Spreadsheet Data Exports
CSV exports are now sanitized against potential formula injection vulnerabilities. This ensures exported data is completely safe to analyze in spreadsheet tools like Excel.
Plus 31 internal changes for stability and performance.