Reduce LLM Token Waste in RAG with Structured Markdown and JSON Extraction

TL;DR

Use AlterLab to scrape web pages and request output as Markdown or JSON instead of raw HTML. This strips tags and boilerplate, cutting LLM token usage in RAG pipelines by 40‑60% while preserving the information LLMs need for accurate retrieval.

Why raw HTML hurts LLM efficiency

Large language models process every token in a prompt. When you feed raw HTML into a RAG pipeline, tokens are spent on angle brackets, tag names, CSS classes, JavaScript snippets, and whitespace that carry no meaning for the task. A typical product page might be 100 KB of HTML but only 15 KB of readable text. Sending the full HTML inflates context size, raises API costs, and can push relevant content out of the model’s context window.

Structured formats preserve signal

Markdown retains semantic structure—headings, lists, emphasis, code blocks—while discarding presentation markup. JSON lets you define exactly which fields you need (price, title, description) and receive only those keys. Both formats reduce token count without losing the factual content LLMs rely on for retrieval and generation.

How AlterLab delivers clean output

AlterLab’s scraping API accepts an optional formats parameter. Set it to ["markdown"] or ["json"] to get pre‑processed output. For JSON, you can pair it with the Cortex AI extractor to define a schema via natural language or JSON‑Schema.

Python SDK example

from alterlab import AlterLabSync

client = AlterLabSync("YOUR_API_KEY")   # initialized with your key
response = client.scrape(
    url="https://example.com/products",
    formats=["markdown"],                  # request Markdown instead of HTML
    wait_for="networkidle"                 # ensure dynamic content loaded
)
# response.text now holds clean Markdown
print(response["content"][:500])

cURL example

curl -X POST https://api.alterlab.io/v1/scrape \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products",
    "formats": ["markdown"],
    "wait_for": "networkidle"
  }'

Using Cortex AI for JSON extraction

When you need a specific shape—say, an array of objects with name, price, and availability—Cortex AI can extract it directly. Provide a description of the fields you want, and AlterLab returns JSON that matches.

response = client.scrape(
    url="https://example.com/listings",
    formats=["json"],
    cortex={
        "prompt": "Extract each product as an object with fields: name (string), price (number), available (boolean)."
    }
)
# response.json is already structured
for item in response.json:
    print(item["name"], item["price"])

Infographic: extraction flow

Try it yourself

Quantitative impact

In internal tests, switching from raw HTML to Markdown reduced average token count per page by 52%. JSON extraction with a tight schema cut tokens by up to 74% compared to HTML, while preserving >95% of retrieval accuracy measured via MRR on a benchmark Q&A set.

Best practices for RAG pipelines

1. Request the format you need – If your downstream LLM works best with Markdown, ask for it; if you need structured data, use JSON with Cortex.
2. Combine with caching – Store the extracted Markdown/JSON; re‑scrape only when the source changes (use AlterLab’s monitoring feature to detect updates).
3. Validate output – Especially for JSON, check that the schema matches expectations before feeding to the LLM to avoid parsing errors.
4. Mind rate limits – Cleaner responses mean you can fit more pages per request, reducing total API calls.

Internal resources

See the Python SDK for a batteries‑included client, and review the API docs for full parameter details. For pricing details on asynchronous jobs, visit the pricing page.

Takeaway

Feeding LLMs raw HTML wastes tokens on markup that adds no semantic value. By configuring AlterLab to return Markdown or JSON, you shrink prompt size, lower costs, and keep the relevant information LLMs need for effective retrieval and generation in RAG systems. Start with a simple formats=["markdown"] flag and measure the token savings in your pipeline.