Changelog

Improved batch scraping interface

Schedule scrapes with cron expressions

Playground now a top-level dashboard item

Apply BYOP discount to scheduled scrapes

Accurate cost estimation for scheduled scrapes

The revert also modified ~80 files' content (not just deleted files). This restores all content-modified files from f0fcbcd0, excluding the 5 files wi

The revert of on main propagated to feature and deleted ~77 files. PRs -986 only restored the 5 files they modified. This restores all rem

Enhanced SSRF protection for scrape targets

Animations now pause when tab is hidden

Lazy-loaded billing chart reduces load time

Prevents infinite loop deadlock

Browsers now close properly on shutdown

Phase 5 cutover: OVH Umami/Cortex containers replaced by E2E admin server instances, exposed through autossh reverse tunnels. - Remove umami + cortex

CI lint failed after PR #861 merge — Black 23.12.1 reformats these files.

Blocking: - Add extra_hosts to all 4 worker containers in docker-compose.prod.yml so workers can resolve host.docker.internal for Cortex telemetry -

- Add fraud_flagged check in /verify-code endpoint before granting welcome credits (billing audit finding) - Add fraud_flagged check in Stripe webho

- Add fraud_flagged check before granting credits in deps.py JWT auto-create flow (was the only remaining unguarded path) - Add /api/v1/webhooks/str

- Guard welcome credits behind fraud_flagged check in both signup paths - Add fraud_flagged_at timestamp to complete_signup and oauth_signin - Cancel

Improved usage page with balance display

Improved homepage performance

Improved search engine optimization

Fixed One Tap authentication issues

Resolved OAuth sign-in errors

Protected against brute-force attempts

Get detailed scrape results in webhooks

View promo bonus details in invoice history

Reduced memory leaks for faster scraping

Redesigned dashboard with improved UI

Clearer error messages for scraping failures

Reject malformed URLs for better scraping

Apply promo codes during checkout

Validate promo codes

Reduced latency

New branded email templates

Hides internal details

Prevents brute-force attacks

Competitors charge a flat rate regardless of site complexity. Our smart router picks the cheapest tier per URL, so the effective average cost is 30-40

Update docstring tier prices in config.py and unified_consumer.py to match current TIER_MICROCENTS (T3=2000, T3.5=2500, T4=4000). Fix ComparisonHero

Worker queues.py had its own TIER_MICROCENTS dict with old T3=500, T4=1000 values while API pricing.py was updated to T3=2000, T4=4000. This caused th

Nav links shifted position between pages because right-side content (search bar, version switcher, theme toggle) varies in width. Using absolute left-

Blog listing page was missing the header entirely, and DocsTopBar was missing the Blog link that MainNav and BlogHeader both include.

T3 (stealth): 500 → 2000 microcents ($0.002/req) T3.5 (lightjs): 700 → 2500 microcents ($0.0025/req) T4 (browser): 4000 microcents ($0.004/req) Cover

Added almaany.com content extraction playbook

Improved performance on non-Cloudflare sites

Improved feedback verification pipeline

Fewer false positives on Cloudflare-protected sites

Improved text/markdown extraction from HTML

Addressed XSS and secret bypass vulnerabilities

Add rich blog content

Improve search engine visibility

Resolve blog post page crashes

Fix unauthenticated usage estimates

Validate window_days param for security

Prevent unauthorized crawler access

Extracts content from any page

Manage blog posts and comments

Securely connect with n8n

Better content extraction

Updated welcome modal

Protects user data

Simpler tier names and layout

Added step progress indicator

Extended to 10 minutes

Run button now works as expected

GA4 now works as expected

Limits OTP attempts to 5

Extract tweet data faster

Extract OG metadata

Optimized browser engine

Improved structured data

Improved dashboard experience

Better protection for user data

Users clicking Copy on the Integration Code and Request tabs were shown the sign-up modal. Code examples are educational content that should be freely

- Fix /pricing schema: use SoftwareApplication instead of Product (correct for SaaS) - Add breadcrumb schemas to /docs, /about, /contact, /changelog l

The js_consumer_v2.py file defined its logger as 'logger' but two log statements in the Penetrator behavior simulation code incorrectly used 'log' ins

Closes integration gaps between Cortex intelligence and scraper orchestrator: - Gap #1: Consolidated proxy decision logic - orchestrator now respects

- Handle loading state in MainNav to prevent "Sign In" → "Dashboard" flicker - Add 5-minute background session refresh interval to keep sessions alive

Track usage with GA4

Easily report scraping issues

Friendly signin messages

Faster page loading

Accurate e-commerce scraping

Protect user data

Improved changelog

Bypass Cloudflare detection

Create and manage support tickets

Added Umami analytics for better insights

Improved performance with reduced delays

Increased free tier limit to 5,000 scrapes

Fixed balance inconsistencies

Faster JSON extraction

Easier access to webhooks

Jobs page removed

Improved speed and stealth

Webhooks link restored

Clickable balance info

Extract content from React/Vue SPA pages

Added mandatory OTP verification for signups

Added search functionality to dashboard docs

Improved performance on WAF-protected sites

Better detection of block pages from WAFs

Added rate limits and OTP verification

Execute commands directly in terminal

Quickly view recent logs on Overview tab

Sortable tables and animations added

Increased font size and better contrast

Cursor and input area issues resolved

Missing image field added to Product schema

New dashboard widget with Fast/Full mode toggle for instant scraping.

Redesigned charts with smoother animations and better visual clarity.

Reduced re-renders for smoother performance and quicker data refreshes.

Updated thresholds and tooltips for clearer performance insights.

Added sortable headers to easily organize and analyze log data.

Resolved issues with undefined stats and improved data handling.

Improved reliability of database updates during migrations.

Corrected false positives for unavailable products on Amazon.

Usage dashboard now correctly shows failed scrapes with 4xx/5xx errors.

Distinguish between playground and API calls for better debugging.

Added clear examples for mapping Amazon product fields to your schema.

Corrected extraction_schema documentation and added Amazon field mapping examples.

Corrected async/await usage in Python SDK documentation.

Escaped special characters in Python examples to prevent build errors.

Corrected response field name from structured_content to filtered_content.

New bot for production monitoring and alerts in Discord.

Improved readability and accessibility of the cookie banner.

Fixed missing field aliases for better playbook compatibility.

Playbook min_tier settings now override Cortex intelligence.

Optimized IP-to-region lookups for significantly improved dashboard performance.

Instant tier upgrades when adding funds, with clearer concurrency limits.

Resolved configuration issues causing service interruptions.

Added canonical URLs, redirects, and robots.txt to improve search engine visibility.

Webhooks and proxy test endpoints now validate URLs to prevent SSRF attacks.

Resolved timing attacks and verification code leaks in authentication flows.

Improved scraping success with Cortex-based anti-detection enhancements.

Added realistic mouse movement patterns for better evasion of detection.

Improved scraping success with Cortex-based anti-detection enhancements.

Added Bezier curve mouse movement for more natural scraping behavior.

Added Cloudflare challenge platform to enhance scraping capabilities.

Reduces false alerts by validating errors only after deployment completion.

Ignores transient issues for 30 minutes post-deployment to avoid noise.

Fixed a field naming issue in API responses to prevent potential conflicts.

Content extraction now returns all parsed data by default for richer results.

Capture all network traffic for improved API learning capabilities.

Users can now opt-in to filtered extraction via the filter_content parameter.

Fixed extraction routing to properly respect the filter_content parameter.

Fixed Cloudflare Worker failures when KV cache is not configured.

Added required proxy secret to prevent unauthorized access.

Marketplace requests now bypass credit checks for seamless external billing.

Integrate with RapidAPI and AWS Marketplace for broader distribution.

Updated CSP rules for Clarity and Google Sign-In compatibility.

Resolved an error that occurred when processing marketplace requests.

Job completion metrics now accurately reflect succeeded status.

Fixed a security issue where billing could be bypassed with marketplace headers.

Automatically generated search index for faster and more accurate documentation searches.

Added comprehensive Node.js SDK docs with API reference and installation guides.

New SDK overview page with badges, GitHub stars, and language-specific links.

Added SDK links to the documentation sidebar for easier access.

Pull requests now target feature branches for better collaboration.

Resolved issue with Clarity analytics tracking not being properly configured.

Prevents jobs from getting stuck due to system failures, ensuring reliable scraping.

Filter extracted data using JSON Schema for more precise results.

New keyword-optimized pages with enhanced designs and trust signals.

Redesigned tables with better alignment and responsive layouts.

Replaced credits with dollar amounts for transparent pricing.

Improved extraction of lazy-loaded images from sites like Squarespace.

New users now receive a welcome email with helpful tips after signing up.

Added a beginner-friendly tutorial to help users make their first API request.

Pricing is now simplified with a consistent tier system across the platform.

Resolved a build issue that caused failures in the pricing calculator.

Easily migrate from Firecrawl with a drop-in API compatibility layer.

New metrics including failure rate, API error rate, and worker status.

Monitoring scripts now dynamically detect active deployment environments.

User-friendly error messages for authentication issues like account linking.

Improved parsing for Claude Haiku output and status extraction.

Enhanced production monitoring with two-tier alerting for better reliability.

Cleaner UI with inline lock icons and tooltips for unavailable features.

Resolved intermittent 500 errors in dashboard stats for smoother performance.

Fixed database connection issues in monitoring to ensure accurate reporting.

Tier 4 browsers now automatically escalate to Penetrator when challenges are detected.

Switched to a more efficient monitoring agent, cutting costs significantly.

Removed excessive console logs for a smoother user experience.

Corrected tier mapping to accurately display current tiers (e.g., Tier 3 as Stealth).

Clarified that caching is opt-in (disabled by default) in the docs.

Reduced GPU workload on mobile devices with optimized shaders and FPS throttling.

Improved decimal precision for sub-cent pricing to ensure accurate cost display.

Fixed an issue where scrapers would remain stopped after a failed deployment.

Added protection against DNS rebinding attacks to prevent SSRF vulnerabilities.

Pricing has been adjusted to $0.0002 per request for Tier 1 and $0.0003 for Tier 2.

Automatically skips lower tiers for domains that require higher tiers, saving 3-5 seconds per request.

Reduces time-to-data for high-protection domains like Walmart from ~10s to 3-5s.

Replaced decimal tiers (0.5, 1.5) with clean integers (1-5) for easier understanding.

Unified scrape endpoint now returns content in text, HTML, and JSON formats.

Parallel IP racing now works with Bring Your Own Proxy (BYOP) users.

Fixed incorrect rate limit values in low balance email alerts.

Workers now use AI-generated playbooks for smarter site-specific scraping.

Reduced GPU overhead by merging visual effects into a single shader.

Enhanced detection and resolution for all challenge types, not just Cloudflare.

Added support for reduced motion and off-screen rendering optimizations.

Fixed scraping blocks on SeatGeek and Ticketmaster by improving detection evasion.

Eliminated 504 errors during worker transitions for smoother operations.

Automatically solves 'Press & Hold' challenges on protected sites.

Forces IP rotation for high-protection domains like Walmart and Amazon.

Fixed an issue where US state targeting returned incorrect IPs due to wrong parameter format.

Resolved random blocking issues on sites like Walmart by stabilizing the browser profile.

Ensures consistent identity and TLS settings for Tier 1 requests.

Fixed 14 vulnerabilities including auth bypass and secret handling.

Added robots.txt to block unnecessary paths and reduce crawl budget waste.

Fixed initial render issues with sidebar width and responsiveness.

Corrected broken links in the API documentation for better navigation.

Resolved inconsistencies between Redis and Postgres for job status tracking.

Automatically fetches a proxy when non-high-risk domains are blocked at Tier 0/1.

Improved deployment security by switching to key-based SSH authentication.

Track organization-level actions with detailed audit logs for better accountability.

View workspace billing statistics directly in the organization dashboard.

Enhanced mobile responsiveness with a new sidebar and responsive header.

Optimized queries and caching for up to 4x faster dashboard loading.

Error messages now show dollar amounts instead of abstract credits.

Updated security policies to allow Lottie animations from trusted CDNs.

Quickly log in with Google One Tap for a seamless authentication experience.

Updated tagline and metrics to highlight LLM-ready output and free scrapes.

Added a dynamic Three.js visualization of the Cortex system on the homepage.

Fixed Google One Tap login by ensuring proper client ID configuration.

Resolved missing Stripe public key configuration for payment processing.

Fixed demo overlay buttons to correctly show login state and actions.

Extract job postings from naukri.com with optimized speed and geo-targeted proxies.

Enhanced thread-safe operations and fail-closed rate limiting for improved stability.

Increased decimal precision to 4 places for accurate sub-cent cost calculations.

Resolved issues with migration file naming to ensure proper validation.

Added fallback handling for unknown job statuses to prevent crashes.

Restricted database access to localhost for enhanced security.

Reduced memory footprint for better performance and stability.

Improved handling of worker processes to prevent memory leaks.

Users with a balance now enjoy unlimited rate limits, while free tier remains at 100 requests/min.

Horizontal scaling with multiple worker replicas improves handling of high-concurrency requests.

Resolved issues with deployments failing due to limited RAM.

Added DNS caching to prevent resolution failures and browser pool exhaustion.

Use either API key, username/password, or both for SmartProxy integration.

Create, edit, and delete blog posts with new admin interfaces.

Improved parsing of SmartProxy.org gateway credentials for smoother integration.

Resolved authentication errors when using country-specific IPs with SmartProxy.org.

Fixed an invalid parameter issue in test results to ensure accurate reporting.

Fixed missing response time in error cases for connectivity tests.

Enjoy unlimited scraping without throttling or restrictions.

Hover over your tier badge to see limits and upgrade progress.

Eliminated delays caused by incorrect proxy configuration.

Concurrency now scales with your balance for flexible usage.

Corrected proxy hostname to prevent immediate DNS errors in Tier 1 scrapes.

Resolved issues with worker health checks in split architectures.

Download PDFs and screenshots directly from the platform.

Users can now enable caching per request with a simple parameter.

Set custom time-to-live (TTL) for cached responses to fit your needs.

PDF generation now skips unnecessary challenge detection for quicker results.

Responses now include cached_at and expires_at fields for better visibility.

PDFs now correctly display blocked pages instead of triggering false challenges.