Guide

Webhooks

Receive real-time notifications when your scraping jobs complete, fail, or require attention. No more polling - let AlterLab push results directly to your server.

Why Use Webhooks?

Real-Time Notifications

Get instant notifications when jobs complete instead of constantly polling the API.

Reduced API Calls

Save on rate limits and API usage by receiving push notifications instead of polling.

Event-Driven Architecture

Build reactive systems that respond immediately to scraping events.

Automatic Retries

AlterLab automatically retries failed deliveries with exponential backoff.

Webhooks vs WebSocket

Webhooks are best for server-to-server communication and batch processing.WebSocket is better for real-time dashboards and client-side updates. You can use both simultaneously.

How Webhooks Work

1. Job Completes

When a scraping job finishes (success or failure), AlterLab checks if you have webhooks configured for that event type.

2. Payload Signed

The event payload is JSON-serialized and signed with your webhook secret using HMAC-SHA256. The signature is included in the X-Alterlab-Signature header.

3. HTTP POST Sent

AlterLab sends an HTTP POST request to your webhook URL with the signed payload. Your server must respond within 10 seconds.

4. Response Handled

A 2xx response indicates success. Any other status code or timeout triggers automatic retries (up to 3 attempts).

Setup Guide

Step 1: Create a Webhook Endpoint

Create an HTTP endpoint on your server that accepts POST requests:

from flask import Flask, request, jsonify
import hmac
import hashlib

app = Flask(__name__)
WEBHOOK_SECRET = "whsec_..."  # Your webhook secret

@app.route("/webhooks/alterlab", methods=["POST"])
def handle_alterlab_webhook():
    # Verify signature
    signature = request.headers.get("X-Alterlab-Signature", "")
    expected_sig = "sha256=" + hmac.new(
        WEBHOOK_SECRET.encode(),
        request.data,
        hashlib.sha256
    ).hexdigest()

    if not hmac.compare_digest(signature, expected_sig):
        return jsonify({"error": "Invalid signature"}), 401

    # Process the event
    event = request.json
    event_type = event.get("event")

    if event_type == "job.completed":
        job_data = event.get("data", {})
        print(f"Job completed: {job_data.get('job_id')}")
        # Process the scraped content...

    elif event_type == "job.failed":
        print(f"Job failed: {event.get('data', {}).get('error')}")

    return jsonify({"received": True}), 200

Step 2: Register the Webhook

curl -X POST https://api.alterlab.io/api/v1/user-webhooks \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Webhook",
    "url": "https://your-server.com/webhooks/alterlab",
    "events": ["job.completed", "job.failed"]
  }'

Save Your Secret!

The webhook secret is only returned once during creation. Save it securely - you'll need it to verify webhook signatures.

Step 3: Test the Webhook

Send a test event to verify your endpoint is working:

curl -X POST https://api.alterlab.io/api/v1/user-webhooks/{webhook_id}/test \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"event_type": "job.completed"}'

Event Types

Event	Description	When Triggered
job.completed	Scraping job finished successfully	Job status changes to "succeeded"
job.failed	Scraping job failed after all retries	Job status changes to "failed"
job.started	Scraping job began processing	Worker picks up the job
batch.completed	All jobs in a batch finished	Last job in batch completes
credits.low	Balance below threshold	Balance drops below 10% of plan
credits.exhausted	Balance depleted	Balance reaches zero

Payload Format

All webhook payloads follow a consistent JSON structure:

job.completed Payload

{
  "event": "job.completed",
  "timestamp": "2025-01-15T10:30:45.123Z",
  "data": {
    "job_id": "550e8400-e29b-41d4-a716-446655440000",
    "batch_id": null,
    "url": "https://example.com/page",
    "status": "succeeded",
    "result": {
      "status_code": 200,
      "content": "<!DOCTYPE html>...",
      "title": "Example Page",
      "metadata": {
        "description": "Page description",
        "keywords": ["example"]
      }
    },
    "billing": {
      "total_credits": 3,
      "tier_used": "3",
      "escalations": [
        {"tier": "1", "result": "failed", "credits": 1},
        {"tier": "3", "result": "success", "credits": 3}
      ]
    },
    "timing": {
      "queued_at": "2025-01-15T10:30:30.000Z",
      "started_at": "2025-01-15T10:30:32.000Z",
      "completed_at": "2025-01-15T10:30:45.123Z",
      "duration_ms": 13123
    }
  }
}

job.failed Payload

{
  "event": "job.failed",
  "timestamp": "2025-01-15T10:31:00.000Z",
  "data": {
    "job_id": "550e8400-e29b-41d4-a716-446655440001",
    "url": "https://protected-site.com",
    "status": "failed",
    "error": {
      "code": "BLOCKED_BY_ANTIBOT",
      "message": "Site blocked access after all tier escalations",
      "details": {
        "last_tier": "4",
        "attempts": 6
      }
    },
    "billing": {
      "total_credits": 0,
      "refunded": true,
      "reason": "All tiers failed"
    }
  }
}

HTTP Headers

Header	Description	Example
X-Alterlab-Signature	HMAC-SHA256 signature of payload	sha256=abc123...
X-Alterlab-Event	Event type	job.completed
X-Alterlab-Delivery	Delivery ID for tracking	del_550e8400...
Content-Type	Always JSON	application/json
User-Agent	AlterLab identifier	AlterLab-Webhook/1.0

Signature Verification

Always Verify Signatures

Never process webhook payloads without verifying the signature first. This prevents attackers from sending fake events to your endpoint.

Verification Steps

Get the raw request body (before JSON parsing)
Get the X-Alterlab-Signature header
Compute HMAC-SHA256 of the body using your webhook secret
Compare the computed signature with the header value
Use constant-time comparison to prevent timing attacks

import hmac
import hashlib

def verify_webhook_signature(payload: bytes, signature: str, secret: str) -> bool:
    """Verify the webhook signature using HMAC-SHA256."""
    expected = "sha256=" + hmac.new(
        secret.encode("utf-8"),
        payload,
        hashlib.sha256
    ).hexdigest()

    # Use constant-time comparison
    return hmac.compare_digest(signature, expected)

# Usage in Flask
@app.route("/webhooks/alterlab", methods=["POST"])
def handle_webhook():
    signature = request.headers.get("X-Alterlab-Signature", "")

    if not verify_webhook_signature(request.data, signature, WEBHOOK_SECRET):
        return jsonify({"error": "Invalid signature"}), 401

    # Safe to process...

API Reference

Create Webhook

POST

/api/v1/user-webhooks

Create a new webhook endpoint. The signing secret is only returned once.

Parameters

Name	Type	Required	Description
name	`string`	Required	Display name for the webhook
url	`string`	Required	HTTPS URL to receive webhook events
events	`string[]`	Required	Event types to subscribe to

Request Example

curl -X POST https://api.alterlab.io/api/v1/user-webhooks \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Webhook",
    "url": "https://your-server.com/webhooks/alterlab",
    "events": ["job.completed", "job.failed"]
  }'

Response Example

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Production Webhook",
  "url": "https://your-server.com/webhooks/alterlab",
  "events": ["job.completed", "job.failed"],
  "is_active": true,
  "failure_count": 0,
  "secret": "whsec_abc123...",  // Only returned once!
  "created_at": "2025-01-15T10:30:00Z"
}

List Webhooks

GET

/api/v1/user-webhooks

List all webhooks for the authenticated user.

Parameters

Name	Type	Required	Description
include_inactive	`boolean`	Optional	Include disabled webhooksDefault: `false`

Request Example

curl -X GET https://api.alterlab.io/api/v1/user-webhooks \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN"

Response Example

{
  "webhooks": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Production Webhook",
      "url": "https://your-server.com/webhooks/alterlab",
      "events": ["job.completed", "job.failed"],
      "is_active": true,
      "failure_count": 0,
      "last_success_at": "2025-01-15T10:30:45Z"
    }
  ],
  "total": 1
}

Test Webhook

POST

/api/v1/user-webhooks/{webhook_id}/test

Send a test event to verify your webhook endpoint is working correctly.

Parameters

Name	Type	Required	Description
webhook_id	`uuid`	Required	Webhook ID to test
event_type	`string`	Optional	Event type to simulateDefault: `job.completed`

Request Example

curl -X POST https://api.alterlab.io/api/v1/user-webhooks/550e8400.../test \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"event_type": "job.completed"}'

Response Example

{
  "success": true,
  "status_code": 200,
  "response_time_ms": 234,
  "error": null
}

Rotate Secret

POST

/api/v1/user-webhooks/{webhook_id}/rotate-secret

Generate a new signing secret. Update your server immediately after rotating.

Parameters

Name	Type	Required	Description
webhook_id	`uuid`	Required	Webhook ID

Request Example

curl -X POST https://api.alterlab.io/api/v1/user-webhooks/550e8400.../rotate-secret \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN"

Response Example

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "secret": "whsec_newSecret123...",  // New secret
  ...
}

View Delivery History

GET

/api/v1/user-webhooks/{webhook_id}/deliveries

View delivery history and debug failed deliveries.

Parameters

Name	Type	Required	Description
webhook_id	`uuid`	Required	Webhook ID
limit	`integer`	Optional	Number of deliveries to returnDefault: `50`
status	`string`	Optional	Filter by status: pending, success, failed, retrying

Request Example

curl -X GET "https://api.alterlab.io/api/v1/user-webhooks/550e8400.../deliveries?limit=10" \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN"

Response Example

{
  "deliveries": [
    {
      "id": "del_123...",
      "event_type": "job.completed",
      "status": "success",
      "attempt_count": 1,
      "max_attempts": 3,
      "response_status_code": 200,
      "response_time_ms": 234,
      "error_message": null,
      "created_at": "2025-01-15T10:30:45Z"
    },
    {
      "id": "del_456...",
      "event_type": "job.completed",
      "status": "failed",
      "attempt_count": 3,
      "max_attempts": 3,
      "response_status_code": 500,
      "response_time_ms": 1200,
      "error_message": "Server returned 500",
      "created_at": "2025-01-15T09:15:00Z"
    }
  ],
  "total": 2
}

Retry Behavior

Attempt	Delay	Total Wait
1 (Initial)	Immediate	0s
2 (First retry)	1 minute	1m
3 (Final retry)	5 minutes	6m

Automatic Disabling

After 5 consecutive failures across multiple events, the webhook is automatically disabled to prevent wasting resources. Re-enable it in the dashboard after fixing the issue.

Best Practices

1. Respond Quickly

Return a 200 response as soon as possible (within 10 seconds). Process the payload asynchronously if needed. A slow response triggers unnecessary retries.

@app.route("/webhooks/alterlab", methods=["POST"])
def handle_webhook():
    # Verify signature first
    if not verify_signature(request):
        return "", 401

    # Queue for async processing
    queue.enqueue(process_webhook, request.json)

    # Return immediately
    return "", 200

2. Handle Duplicates

Due to retries, you may receive the same event multiple times. Use thejob_id or delivery ID to deduplicate events.

3. Use HTTPS

Webhook URLs must use HTTPS. HTTP endpoints are not supported for security reasons.

4. Rotate Secrets Periodically

Rotate your webhook secret every few months. Use the rotate-secret endpoint and update your server configuration immediately.

5. Monitor Delivery Stats

Check the webhook detail endpoint periodically to monitor success rates and catch issues early.

Webhook Limits

Maximum webhooks per user: 10
Request timeout: 10 seconds
Maximum retries: 3 attempts
Auto-disable threshold: 5 consecutive failures
Payload size limit: 1MB

← Previous: BYOP Guide Next: Error Codes →