AI Agent & MCP Integration

npm install -g alterlab-mcp-server

{
  "mcpServers": {
    "alterlab": {
      "command": "npx",
      "args": ["-y", "alterlab-mcp-server"],
      "env": {
        "ALTERLAB_API_KEY": "sk_live_your_api_key_here"
      }
    }
  }
}

You are a research agent with web scraping capabilities.

When asked to research a topic:

1. Plan: Identify 3-5 authoritative sources to investigate
2. Scrape: Use alterlab_scrape to fetch each source as markdown
3. Extract: If pages have structured data, use alterlab_extract
4. Verify: Cross-reference facts across multiple sources
5. Synthesize: Produce a structured report with citations

Rules:
- Always check costs with alterlab_estimate_cost before large batches
- Use alterlab_check_balance to monitor credit usage
- Prefer markdown format for scraping (best for analysis)
- Use render_js=true only for JavaScript-heavy sites (SPAs, dashboards)
- Cite sources with URLs in your final report

User: Research the current state of web scraping regulations
      in the EU. Include recent court rulings.

Agent: I'll research this topic across multiple sources.

[Calls alterlab_scrape: "https://gdpr.eu/tag/web-scraping/"]
[Calls alterlab_scrape: "https://techcrunch.com/tag/web-scraping/"]
[Calls alterlab_scrape: "https://eur-lex.europa.eu/..." with render_js=true]
[Calls alterlab_extract on legal database with extraction_prompt:
 "Extract case name, date, ruling, and implications for web scraping"]

Agent: Here's my research report:

## EU Web Scraping Regulations - March 2026

### Key Findings
1. GDPR Article 6(1)(f) — legitimate interest basis...
2. Recent CJEU ruling in Case C-xxx/24...
3. ...

### Sources
- [GDPR.eu](https://gdpr.eu/...) — Retrieved March 2026
- [TechCrunch](https://techcrunch.com/...) — Retrieved March 2026
- [EUR-Lex](https://eur-lex.europa.eu/...) — Official text

import alterlab
import json

client = alterlab.AlterLab(api_key="sk_live_your_key")

def research_agent(topic: str, max_sources: int = 5) -> dict:
    """Autonomous research agent that scrapes and synthesizes."""

    # Step 1: Scrape a search-oriented page for source discovery
    search_result = client.scrape(
        url=f"https://news.ycombinator.com/",
        formats=["markdown"],
    )

    # Step 2: Scrape each source and collect content
    sources = []
    for url in discovered_urls[:max_sources]:
        # Check cost first
        estimate = client.estimate(url=url)
        print(f"Estimated cost for {url}: {estimate.credits} credits")

        result = client.scrape(
            url=url,
            formats=["markdown"],
            advanced={"render_js": estimate.needs_js},
        )
        sources.append({
            "url": url,
            "content": result.markdown[:5000],  # Trim for context window
            "title": result.metadata.get("title", ""),
        })

    # Step 3: Extract structured data where applicable
    for source in sources:
        extraction = client.scrape(
            url=source["url"],
            formats=["json"],
            extraction_prompt=f"Extract key facts about {topic}",
        )
        source["structured_data"] = extraction.json_data

    return {
        "topic": topic,
        "source_count": len(sources),
        "sources": sources,
    }

When I ask a question, follow this process:

1. Identify 2-3 authoritative URLs that would answer the question
2. Use alterlab_scrape to fetch each URL as markdown
3. Read the scraped content carefully
4. Answer my question using ONLY information from the scraped pages
5. Cite each claim with the source URL

Always scrape fresh content — don't rely on your training data for
facts that may have changed.

import alterlab
from openai import OpenAI

scraper = alterlab.AlterLab(api_key="sk_live_your_key")
llm = OpenAI()

def rag_answer(question: str, source_urls: list[str]) -> str:
    """Answer a question using fresh web content as context."""

    # Step 1: Scrape all source URLs
    context_parts = []
    for url in source_urls:
        result = scraper.scrape(
            url=url,
            formats=["markdown"],
        )
        context_parts.append(
            f"## Source: {url}\n\n{result.markdown[:3000]}"
        )

    context = "\n\n---\n\n".join(context_parts)

    # Step 2: Send to LLM with scraped context
    response = llm.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": (
                    "Answer questions using ONLY the provided context. "
                    "Cite sources with URLs. If the context doesn't "
                    "contain the answer, say so."
                ),
            },
            {
                "role": "user",
                "content": f"Context:\n{context}\n\nQuestion: {question}",
            },
        ],
    )

    return response.choices[0].message.content

# Usage
answer = rag_answer(
    question="What are the latest changes to robots.txt standards?",
    source_urls=[
        "https://developers.google.com/search/docs/crawling-indexing/robots-txt",
        "https://www.rfc-editor.org/rfc/rfc9309",
    ],
)
print(answer)