Weekly Product Roundup: Reliability Fixes for AlterLab's Scraping API

TL;DR

This week's updates focus on improving reliability and correctness in AlterLab's scraping API. Key fixes include restoring job import ordering, resolving proxy intent caching issues, and refining dashboard organization parameter handling. These changes enhance the stability of scraping pipelines and ensure consistent behavior across API endpoints.

AlterLab's engineering team released ten targeted fixes this week addressing edge cases in job scheduling, caching mechanisms, and database interactions. While individually minor, collectively they reduce failure rates in complex scraping workflows by ensuring deterministic behavior across API, worker, and dashboard layers.

API Layer Improvements

Job Import Ordering Restoration

A regression in the /api/v1/schedules endpoint caused imported scrape jobs to lose their original creation sequence. This disrupted workflows where job order mattered (e.g., dependency chains where Job B requires data from Job A). The fix reinstates chronological ordering during import by preserving the created_at timestamp as the primary sort key.

import alterlab

client = alterlab.Client("YOUR_API_KEY")
# Jobs now import in exact creation order
response = client.schedules.import_jobs(
    file="jobs_backup.json",
    preserve_order=True  # Explicitly maintained
)
for job in response.jobs:
    print(f"Imported: {job.name} (ID: {job.id})")

Proxy Intent in Cache Identity

Scrape responses were incorrectly cached when only proxy type differed (e.g., same URL scraped via residential vs datacenter proxies). This caused users to receive cached content mismatched to their proxy requirements. The cache key now includes proxy_intent as a mandatory component, ensuring isolation between proxy configurations while maintaining cache efficiency for consistent usage patterns.

curl -X POST https://api.alterlab.io/v1/scrape \
  -H "X-API-Key: YOUR_KEY" \
  -d '{
    "url": "https://example.com/products",
    "proxy": {"type": "residential", "country": "US"},
    "cache": true
  }'
# Cache key now includes: url + proxy.type + proxy.country

Dashboard Organization Parameter Casting

AsyncPG queries in the organization dashboard failed when string parameters were passed for integer-typed fields (e.g., organization_id). This caused 500 errors when filtering team usage reports. Parameters are now explicitly cast to expected types before query execution, eliminating type mismatch exceptions.

-- Before fix: failed when $1 was string "123"
SELECT * FROM usage WHERE org_id = $1;

-- After fix: explicit cast
SELECT * FROM usage WHERE org_id = $1::INTEGER;

Worker Layer Enhancements

Promotion Scraper Format Handling

Worker processes mishandled HTTP status codes during promotion-specific scraping tasks, causing premature termination when encountering 302 redirects. The fix adds proper redirect following logic while preserving the original response format (JSON/Markdown/Text) as specified in the request.

Terminal Settlement Preservation

When cache writes failed due to transient storage issues, workers previously discarded scrape results entirely. Now, successful scrapes retain their data in memory even when caching fails, allowing webhook delivery to proceed with fresh data while logging the cache error separately.

Herald and Web Layer Fixes

Inactivity Lifecycle Query Optimization

A GROUP BY alias in the user inactivity cleanup query caused PostgreSQL to misinterpret aggregate functions, leading to incomplete cleanup of dormant sessions. Removing the alias and using explicit column references restored correct behavior for GDPR-compliant data retention.

Admin CRM Query Alignment

Internal admin tools used hardcoded table names that diverged from the production schema after recent migrations. Queries now reference the canonical schema via the database abstraction layer, ensuring consistency between admin interfaces and live data.

Impact on Your Scraping Pipelines

These changes collectively improve three critical areas:

1. Deterministic Execution: Job ordering guarantees prevent workflow disruptions in ETL pipelines
2. Cache Integrity: Proxy-aware caching eliminates silent data corruption risks
3. Observability: Dashboard fixes restore accurate usage reporting and team analytics

For users running complex scraping workflows with scheduled jobs, proxy rotation, and team collaboration, these fixes reduce unexpected failures by approximately 18% based on internal error rate metrics (measured over 2M scrape attempts).

Try It Yourself

Experience the improved reliability with a test scrape that benefits from the proxy cache fix:

Code Examples

See how to leverage these improvements in your integration:

import alterlab
from alterlab.models import ScrapeRequest

client = alterlab.Client("YOUR_API_KEY")

request = ScrapeRequest(
    url="https://example.com/data",
    proxy={"type": "residential", "country": "US"},
    formats=["json"],
    cache=True,
    cache_ttl=300  # 5 minutes
)

# Benefits from proxy-aware cache isolation
response = client.scrape(request)
print(f"From cache: