Bug Fixes & Stability
This release includes 1 update focused on bug fixes & stability.
Custom playbook for moorescoleraine.com that extracts structured JSON with text content per section and images grouped by page section. Handles hero carousels, department pages, brand pages, reviews, and opening hours. Extracts both <img> tags and CSS background images.
New Features
1Add Moores of Coleraine custom playbook
Custom playbook for moorescoleraine.com that extracts structured JSON with text content per section and images grouped by page section. Handles hero carousels, department pages, brand pages, reviews, and opening hours. Extracts both <img> tags and CSS background images.
Improvements
1Content(blog): add 3 SEO-targeted blog posts
Add migration with 3 comprehensive blog posts targeting high-intent scraping keywords: 1. Web Scraping API Comparison 2026 - competitor comparison 2. How to Scrape Without Getting Blocked - educational guide 3. Best Web Scraping API for Developers 2026 - commercial intent Each post includes code ex
Bug Fixes
14Harden WAF AI crawler UA matching patterns
Use version-prefix patterns (e.g. "GPTBot/" instead of "GPTBot") to reduce spoofing surface. Add documentation noting UA spoofing is inherent and listing crawler IP ranges for future hardening.
Add structlog warnings for spend counter drift
Add structlog warnings for spend counter drift implementation
Scope tier clamping to tier-intelligence overrides only (...
Playbook-driven skip_to_tier4 (playbook_requires_js) has its own handler downstream that returns a clear "JS rendering required" error. The clamping logic must only apply to tier-intelligence overrides, otherwise it would bypass the playbook's specific error message.
Clamp tier intelligence overrides when optimal_tier excee...
When tier intelligence determined a domain needs scraping tier+ but the user's plan capped at max_tier=2, the skip_to_tier3 flag set needs_proxy=True which bypassed the scraping tier/2 parallel race block entirely. This caused instant 0-tier failures (~0.01s) with a generic error message, wasting qu
Patch nodriver Cookie.from_json sameParty KeyError
Chrome removed the 'sameParty' cookie attribute from CDP responses, but nodriver <= 0.48.1 still uses direct dict access json['sameParty'] in Cookie.from_json, causing a KeyError during cookie extraction. Adds a monkey-patch module that replaces Cookie.from_json with a safe version using .get('same
Add deadlock retry to credit and spend-limit functions (#...
Concurrent batch requests from the same user trigger PostgreSQL deadlocks (40P01) due to lock-ordering conflicts between spend-limit reservation (api_key_limits → account_spend_limits) and credit ledger operations (credit_ledger → credit_balance_cache). No retry logic existed, so deadlocks bubbled u
Restore catch-all /api/v1/ rewrite in next.config.js
The narrowed rewrites (scrape + batch only) broke /api/v1/auth/* paths that transit server → Next.js but have no filesystem route handler. The specific rewrites were also redundant since Next.js filesystem routes at /api/v1/scrape/ and /api/v1/batch/ take precedence over rewrites.
Handle list values in secret template extraction
_extract_referenced_secrets now recursively scans strings, dicts, and lists — matching the traversal depth of resolve_config_templates in the worker. Prevents missed secrets when templates appear in list-typed config values.
Use X-Internal-Secret header for internal scrape calls (#...
Replace fabricated X-Internal-Auth header with the proper X-Internal-Secret mechanism defined in . Both scrape_preview and discover_site tools now read API_INTERNAL_SECRET from settings for authenticated internal calls.
Use SessionUser instead of CurrentUser for dashboard-only...
Use SessionUser instead of CurrentUser for dashboard-only... implementation
Optimize meta titles and descriptions for US market CTR (...
Optimize meta titles and descriptions for US market CTR (... implementation
Remove interactive hover affordances from non-clickable e...
Remove hover effects (translateY, scale, shadow, group-hover color changes) from elements that look clickable but have no onClick/href. This reduces dead clicks (16.7% of sessions affected) by making non-interactive elements visually distinct from clickable ones. - CortexSection: remove hover:bg, g
Track signup completion for all auth methods
Move auth-signup-complete analytics event from SignInDialog (email-only) to WelcomeGate (fires on first dashboard visit for every new user). This ensures SSO signups (Google/GitHub/Microsoft) are tracked alongside email signups, fixing the 87% tracking gap (10 tracked vs 76 actual).
Revert hero headline and subtitle to original copy
Revert hero headline and subtitle to original copy implementation