Enforcing Single Deadline and Typed Capacity Outcomes in AlterLab's T4 Worker

TL;DR

AlterLab’s T4 worker now enforces a single monotonic deadline for each logical scrape operation and returns typed capacity outcomes. This change cuts failure latency from 90‑180 seconds to under a minute and prevents browser‑acquisition errors from triggering unnecessary tier or proxy model retrains.

Problem: The Old T4 Behavior

Previously, a single logical T4 operation could spawn multiple outer browser attempts. Each attempt received its own timeout budget, and inner waits—for solver, advanced browser, navigation, redirect, or acquisition—were given fresh timeouts as well. When any of these waits failed, the error was flattened into a generic T4 failure. The system would then retry the operation, which caused the timeout to reset and allowed the job to accumulate time across attempts. In production, CyclingFlash jobs often spent 90–180 seconds before finally failing, wasting compute and delaying feedback.

Additionally, browser‑acquisition errors (e.g., unable to secure a headless browser slot) were treated the same as ordinary scrape failures. This caused the tier‑selection and proxy‑models to be retrained on noisy data, leading to sub‑optimal proxy choices and extra retries that did not address the real capacity constraint.

Solution: One Deadline, Typed Outcomes

The update introduces two core changes:

1. Monotonic deadline – When a T4 operation begins, a single deadline is recorded based on the configured timeout (e.g., 60 seconds). All inner waits and retries share this deadline; they do not receive refreshed timeouts. If the deadline passes, the operation fails immediately, regardless of how many inner attempts have been made.

2. Typed capacity outcomes – The worker now distinguishes between:

   • Scrape failures (e.g., HTTP 404, parsing errors, solver timeout)
   • Capacity failures (e.g., browser‑acquisition timeout, proxy‑pool exhaustion)

   Capacity failures are returned with a distinct error type and are not fed into the tier‑proxy training loop. Instead, they trigger a back‑off on the worker pool or signal the scheduler to wait for capacity to free up.

These changes ensure that a logical operation respects the user‑specified timeout ceiling and that capacity signals are handled separately from content‑level errors.

How It Works Under the Hood

When a request hits the /api/v1/scrape endpoint with tier: 4, the orchestration layer creates a T4Job object. The job’s constructor captures start = monotonic_now() and computes deadline = start + timeout_seconds. Every asynchronous step—browser launch, navigation, solver invocation, redirect follow—checks monotonic_now() < deadline before proceeding. If the check fails, the step aborts with a DeadlineExceeded error that bubbles up as the job’s final result.

Capacity checks occur before browser acquisition. If the internal semaphore for browser slots cannot be obtained within the remaining time, the job returns a CapacityExhausted error with a retry_after hint derived from the semaphore’s wait queue. The API layer surfaces this as a distinct error code (429 Capacity) rather than a generic 500.

The tier‑proxy model update subsystem now subscribes only to events with error type ScrapeFailure. Capacity events are routed to a separate metrics stream that informs autoscaling decisions but does not affect model weights.

Impact on Performance and Cost

Latency

• Before: 90–180 seconds to observe a failure (due to accumulated timeouts).
• After: Failure observed within the configured timeout (e.g., 60 seconds) plus a small overhead for cleanup (< 2 seconds).
• Result: Up to 70 % reduction in wasted time for failing jobs.

Compute Usage

Because jobs stop sooner, the average CPU‑second per failed scrape drops proportionally. For a workload with 10 % failure rate, this translates to roughly 0.7 CPU‑seconds saved per scrape.

Proxy Model Stability

Removing capacity‑induced noise from the training set reduces variance in proxy‑score updates by ~15 % (based on internal A/B tests over two weeks). This yields more consistent success rates across retries and lowers the frequency of proxy‑pool churn.

User Experience

Developers receive faster feedback when a target site is truly unreachable or when the platform is at capacity. The distinct error types allow client‑side logic to differentiate between “try again later” (capacity) and “fix your selector or URL” (scrape error).

Code Example: Configuring the Timeout

The following Python snippet shows how to set a 45‑second deadline for a T4 scrape and handle the two possible error types.

import alterlab
from alterlab.exceptions import DeadlineExceeded, CapacityExhausted

client = alterlab.Client("YOUR_API_KEY")  # highlighted

try:
    resp = client.scrape(
        url="https://example.com/products",
        tier=4,
        params={"timeout": 45}  # highlighted
    )
    print(resp.json())
except DeadlineExceeded:
    # The operation used the full timeout without success
    print("Scrape timed out after 45 s")
except CapacityExhausted as exc:
    # Platform lacked a free browser slot; retry_after suggests delay
    print(f"Capacity full, retry after {exc.retry_after}s")

The timeout parameter directly sets the monotonic deadline. If the