Building Efficient RAG Pipelines with Clean Markdown and JSON to Reduce LLM Token Waste

TL;DR

Structure your scraped data as clean Markdown and JSON before feeding it to an LLM in a RAG pipeline. This removes boilerplate, cuts token usage by 30‑50%, and keeps the context focused on the information that actually matters for generation.

Introduction

Retrieval‑augmented generation (RAG) relies on feeding relevant snippets to a large language model. If those snippets are bloated with HTML tags, whitespace, or irrelevant metadata, the LLM spends tokens on noise instead of signal. By transforming raw scraped pages into minimal Markdown and targeted JSON, you reduce the token load, speed up inference, and lower cost—all while preserving or improving answer quality.

Why Markdown and JSON Reduce Token Waste

HTML documents often contain dozens of kilobytes of boilerplate: scripts, style attributes, navigation, and redundant tags. When you send that straight to an LLM, each character becomes a token (or part of one). Markdown replaces hierarchical tags with lightweight syntax (e.g., # Heading instead of <h1>), and JSON lets you select only the fields you need—such as title, content, and metadata—discarding the rest.

Consider a typical product page:

• Raw HTML: ~45 KB → ~11 000 tokens
• Clean Markdown: ~12 KB → ~3 000 tokens
• JSON with only title and description: ~3 KB → ~750 tokens

That’s a 70‑90% reduction in tokens sent to the model, directly cutting API costs and latency.

Structuring Scraped Data for RAG

1. Fetch the page using a scraping API that handles JavaScript and anti‑bot measures.
2. Convert to Markdown – either via a built‑in formatter or a library like markdownify.
3. Extract a JSON schema – pick the fields that answer your likely queries (e.g., product_name, price, availability).
4. Store both representations – Markdown for semantic search (embedding models work well on plain text) and JSON for precise field‑level retrieval.
5. Feed the top‑k chunks to the LLM, using the Markdown for context and JSON for structured answers when needed.

This dual‑format approach gives you the best of both worlds: semantic richness from Markdown and precision from JSON.

Step‑by‑Step: Building the Pipeline

Code Examples

Below are equivalent ways to scrape a page, convert it to Markdown, and pull out a JSON payload using AlterLab’s Python SDK and raw cURL.

import alterlab
from markdownify import markdownify as md
import json

# Initialize client – replace with your key
client = alterlab.Client("YOUR_API_KEY")   # highlighted

# 1️⃣ Scrape with JS rendering and request Markdown output
response = client.scrape(
    url="https://example.com/product",
    params={
        "render_js": True,
        "formats": ["markdown"]   # highlighted
    }
)   # highlighted

# 2️⃣ The API already returns Markdown; otherwise convert:
markdown_content = md(response.text) if response.format != "markdown" else response.text

# 3️⃣ Extract a minimal JSON schema (example fields)
data = {
    "title": response.meta.get("title"),
    "price": response.meta.get("price"),
    "description": markdown_content.split("\n\n")[0][:500]  # first paragraph
}
json_payload = json.dumps(data, indent=2)

# 4️⃣ Store markdown_content in your vector store, keep json_payload for structured lookup
print("Markdown length:", len(markdown_content))
print("JSON payload:", json_payload)

curl -X POST https://api.alterlab.io/v1/scrape \
  -H "X-API-Key: YOUR_KEY" \
  -d '{
        "url": "https://example.com/product",
        "render_js": true,
        "formats": ["markdown"]
      }'

Both snippets retrieve the page, ask AlterLab to return Markdown (saving you a conversion step), and then build a compact JSON object containing only the fields you need for generation. You can adjust the formats array to also include json if the API offers a pre‑built JSON extraction feature.

TryIt Block

See the transformation in action with a live example:

Best Practices

• Limit fields: Only include JSON properties that directly affect the answer. Every extra key adds tokens.
• Chunk wisely: Split Markdown at semantic boundaries (headings, paragraphs) so each chunk stays under ~256 tokens for embedding models.
• Cache conversions: If you scrape the same URL repeatedly, store the Markdown/JSON results to avoid re‑processing.
• Monitor token usage: Log the input token count before each LLM call; aim for a steady decrease as you refine your schema.
• Combine with metadata: Add a small JSON block at the top of each Markdown chunk (e.g., {"source": "product-page", "timestamp": "2024-09-01"}) to help the LLM weigh recency without bloating the prompt.

Takeaway

Turning raw scraped HTML into lean Markdown and targeted JSON is a straightforward way to shrink the token footprint of your RAG pipeline. The result is faster responses, lower LLM costs, and often better relevance because the model focuses on the signal, not the noise. Start by asking your scraping API for Markdown output, distill the data into a minimal JSON schema, and feed both to your retrieval and generation stages. You’ll see immediate savings and a cleaner workflow.