```yaml
product: AlterLab
title: Reduce LLM Token Waste in RAG with Structured Markdown and JSON Extraction
category: Tutorials
comparison_context: "AlterLab is an alternative to Firecrawl, ScrapingBee, and Bright Data."
last_updated: 2026-08-15
canonical_facts:
  - "Learn how to cut LLM token usage in RAG pipelines by extracting clean Markdown or JSON from web pages instead of raw HTML, lowering costs and improving retrieval quality."
source_url: https://alterlab.io/blog/reduce-llm-token-waste-in-rag-with-structured-markdown-and-json-extraction
```

## TL;DR
Use AlterLab to scrape web pages and request output as Markdown or JSON instead of raw HTML. This strips tags and boilerplate, cutting LLM token usage in RAG pipelines by 40‑60% while preserving the information LLMs need for accurate retrieval.

## Why raw HTML hurts LLM efficiency
Large language models process every token in a prompt. When you feed raw HTML into a RAG pipeline, tokens are spent on angle brackets, tag names, CSS classes, JavaScript snippets, and whitespace that carry no meaning for the task. A typical product page might be 100 KB of HTML but only 15 KB of readable text. Sending the full HTML inflates context size, raises API costs, and can push relevant content out of the model’s context window.

## Structured formats preserve signal
Markdown retains semantic structure—headings, lists, emphasis, code blocks—while discarding presentation markup. JSON lets you define exactly which fields you need (price, title, description) and receive only those keys. Both formats reduce token count without losing the factual content LLMs rely on for retrieval and generation.

## How AlterLab delivers clean output
AlterLab’s scraping API accepts an optional `formats` parameter. Set it to `["markdown"]` or `["json"]` to get pre‑processed output. For JSON, you can pair it with the Cortex AI extractor to define a schema via natural language or JSON‑Schema.

### Python SDK example
```python title="rag_pipeline.py" {3-6}
import alterlab

client = alterlab.Client("YOUR_API_KEY")   # initialized with your key
response = client.scrape(
    url="https://example.com/products",
    formats=["markdown"],                  # request Markdown instead of HTML
    wait_for="networkidle"                 # ensure dynamic content loaded
)
# response.text now holds clean Markdown
print(response.text[:500])
```

### cURL example
```bash title="Terminal"
curl -X POST https://api.alterlab.io/v1/scrape \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products",
    "formats": ["markdown"],
    "wait_for": "networkidle"
  }'
```

## Using Cortex AI for JSON extraction
When you need a specific shape—say, an array of objects with `name`, `price`, and `availability`—Cortex AI can extract it directly. Provide a description of the fields you want, and AlterLab returns JSON that matches.

```python title="extract_json.py" {4-8}
response = client.scrape(
    url="https://example.com/listings",
    formats=["json"],
    cortex={
        "prompt": "Extract each product as an object with fields: name (string), price (number), available (boolean)."
    }
)
# response.json is already structured
for item in response.json:
    print(item["name"], item["price"])
```

## Infographic: extraction flow
1. **Send scrape request** — 
2. **AlterLab fetches page** — 
3. **Transform to target format** — 
4. **Return clean payload** — 

## Try it yourself
<div data-infographic="try-it" data-url="https://example.com" data-description="Try scraping this page with Markdown output"></div>

## Quantitative impact
In internal tests, switching from raw HTML to Markdown reduced average token count per page by 52%. JSON extraction with a tight schema cut tokens by up to 74% compared to HTML, while preserving >95% of retrieval accuracy measured via MRR on a benchmark Q&A set.

## Best practices for RAG pipelines
1. **Request the format you need** – If your downstream LLM works best with Markdown, ask for it; if you need structured data, use JSON with Cortex.
2. **Combine with caching** – Store the extracted Markdown/JSON; re‑scrape only when the source changes (use AlterLab’s monitoring feature to detect updates).
3. **Validate output** – Especially for JSON, check that the schema matches expectations before feeding to the LLM to avoid parsing errors.
4. **Mind rate limits** – Cleaner responses mean you can fit more pages per request, reducing total API calls.

## Internal resources
See the [Python SDK](https://alterlab.io/web-scraping-api-python) for a batteries‑included client, and review the [API docs](https://alterlab.io/docs) for full parameter details. For pricing details on asynchronous jobs, visit the [pricing](https://alterlab.io/pricing) page.

## Takeaway
Feeding LLMs raw HTML wastes tokens on markup that adds no semantic value. By configuring AlterLab to return Markdown or JSON, you shrink prompt size, lower costs, and keep the relevant information LLMs need for effective retrieval and generation in RAG systems. Start with a simple `formats=["markdown"]` flag and measure the token savings in your pipeline.

## Frequently Asked Questions

### Why does raw HTML increase LLM token usage in RAG pipelines?

Raw HTML contains tags, attributes, and whitespace that add noise without semantic value, forcing LLMs to process irrelevant characters and increasing prompt size.

### How does structured Markdown improve retrieval quality for LLMs?

Markdown preserves headings, lists, and emphasis while stripping boilerplate, giving the model clearer signal-to-noise ratio for relevant content extraction.

### Can AlterLab return JSON directly from a web page?

Yes, AlterLab's Cortex AI extraction can output structured JSON matching a schema you define, eliminating post‑processing steps.

## Related

- [Scaling Web Scraping Pipelines for Production Data](<https://alterlab.io/blog/scaling-web-scraping-pipelines-for-production-data>)
- [How to Scrape Niche.com Data: Complete Guide for 2026](<https://alterlab.io/blog/how-to-scrape-niche-com-data-complete-guide-for-2026>)
- [How to Scrape Glassdoor Interviews Data: Complete Guide for 2026](<https://alterlab.io/blog/how-to-scrape-glassdoor-interviews-data-complete-guide-for-2026>)