```yaml product: AlterLab title: Ensuring Backup Integrity: Immutable Logical Contracts and Production Table Classification at AlterLab category: Product Updates comparison_context: "AlterLab is an alternative to Firecrawl, ScrapingBee, and Bright Data." last_updated: 2026-09-06 canonical_facts: - "Learn how AlterLab preserves historical logical contracts, validates backup generations, and classifies production recovery tables to guarantee reliable restores for your scraping pipelines." source_url: https://alterlab.io/blog/ensuring-backup-integrity-immutable-logical-contracts-and-production-table-classification-at-alterlab ``` ## TL;DR AlterLab now stores an immutable snapshot of its backup policy and redaction rules with every logical backup generation. When validating a backup or performing a restore, the system compares the data against that generation’s specific contract rather than the latest policy. Additionally, all production‑recovery tables are classified and, where needed, fields like payloads in `jobs_archive` are redacted to ensure restores are both accurate and safe. ## Introduction AlterLab’s scraping API generates a continuous stream of job metadata, scrape results, and usage logs. To protect this data we maintain a nightly backup pipeline that creates logical copies of the underlying store. Early in 2024 we noticed a recurring issue: whenever the backup policy or redaction rules were updated, previously valid backup generations would fail validation. This blocked the promotion of new backups and created anxiety around restore reliability. The solution was two‑fold: preserve the exact contract that governed each backup, and formally classify which production tables belong in the backup set. ## The Problem: Backup Drift and Policy Evolution Our backup system originally validated each retained generation by comparing its contents to the *current* backup‑policy.yaml and redaction‑registry.json files. Any change—even a harmless comment or a new optional field—altered the file hashes and caused the validation step to reject older generations. Because the validation failure prevented the next backup from being published, a single policy tweak could halt the entire nightly pipeline. At the same time, we observed that certain tables were being retained unintentionally. Tables such as `credit_balance_cache_snap_20260725` (a temporary cache) and `jobs_archive` (which stores verbose job payloads) were not part of the intended backup set. Their presence increased backup size and introduced noise during restore validation, yet the live‑schema gate had no way to distinguish them from essential tables. ## Solution Overview: Immutable Logical Contracts ### What is a Logical Contract? A logical contract is a version‑controlled snapshot of two files: - `backup-policy.yaml` – defines which tables, columns, and retention rules apply. - `redaction-registry.json` – lists fields that must be stripped or transformed before storage. When a backup generation is created, we compute a hash of these files and store the hash alongside the backup manifest. The manifest also records the exact versions of the policy and redaction files used. ### How We Store Snapshots The backup workflow now includes an extra step before uploading a generation: 1. Read the live `backup-policy.yaml` and `redaction-registry.json`. 2. Compute SHA‑256 hashes for each. 3. Write the hashes into the generation’s metadata block. 4. Upload the data payload and the metadata block together. Because the contract is immutable, future policy edits do not alter the hash stored with an existing generation. ## Validating Retained Generations Against Historical Contracts ### The Validation Process When the retention job runs, it does **not** compare the generation to the current policy files. Instead: 1. Load the generation’s manifest to retrieve the stored policy and redaction hashes. 2. Retrieve the exact versions of those files from our internal version store (a simple append‑only log). 3. Run the validation checks against those retrieved versions. 4. If the checks pass, the generation is considered valid for the next retention cycle. This approach means that a policy change only affects newly created generations. Older generations continue to validate against the rules that were in place when ## Frequently Asked Questions ### What is a logical contract in the context of AlterLab's backup system? A logical contract is an immutable snapshot of the backup policy and redaction registry that defines how data should be stored and validated. Each backup generation is tied to the contract that was active when it was created, ensuring restores are checked against the exact rules that applied at backup time. ### How does AlterLab prevent backup validation failures when policies evolve? Instead of validating retained generations against the current policy, AlterLab checks each generation against its own historical contract. This decouples backup validity from future policy changes, allowing normal updates without breaking older backups. ### Why does AlterLab redact certain fields in the jobs_archive table during backup? The jobs_archive table may contain transient payloads, results, or error details that are not needed for long-term recovery and could expose internal state. Redacting these fields ensures the backup retains only essential, production-shaped data while maintaining strict restore correctness. ## Related - [How to Scrape JavaScript-Heavy Sites Without Getting Blocked]() - [Fixing PerimeterX Routing Regression in AlterLab's Anti-Bot Stack]() - [Migrate from Subscription to Pay-As-You-Go Scraping API]()