```yaml
product: AlterLab
title: Building Agentic Web Browsing Tools with Real-Time Data and MCP Servers
category: Tutorials
comparison_context: "AlterLab is an alternative to Firecrawl, ScrapingBee, and Bright Data."
last_updated: 2026-08-16
canonical_facts:
  - "Learn how to combine LLM tool use, real-time web data, and MCP servers to create agentic browsing agents that fetch and act on live information without custom scrapers."
source_url: https://alterlab.io/blog/building-agentic-web-browsing-tools-with-real-time-data-and-mcp-servers
```

## TL;DR
Agentic web browsing tools let LLMs autonomously retrieve and act on live web data by treating a scraping API as a callable tool via an MCP server. This approach removes the need for custom scraper code while giving agents up-to-date information for decision-making.

## Introduction
Modern LLM applications often need current facts that lie outside the model’s training data. Instead of relying on stale knowledge or frequent retraining, you can equip the model with a tool that fetches fresh web content on demand. By wrapping a scraping API in an MCP (Model Context Protocol) server, you expose a standardized interface that the LLM can invoke just like any other function. The result is an agent that reasons, decides when to scrape, extracts the needed data, and continues its workflow—all without you writing brittle scraper logic.

## Why Agentic Browsing Matters
Static prompts limit LLMs to the information they were trained on. When a user asks for the latest price, a recent news headline, or up-to-date product specifications, the model must consult an external source. Building agentic browsing solves this by:
- Enabling real‑time lookups without manual data pipelines
- Keeping the scraping logic centralized and maintainable
- Allowing the LLM to control frequency and scope based on context
- Reducing engineering overhead because the scraping API handles anti‑bot measures, JavaScript rendering, and proxy rotation

## Architecture Overview
The system consists of three loosely coupled pieces:
1. **LLM Agent** – decides when to invoke the scraping tool and processes the returned data
2. **MCP Server** – wraps the scraping API in a JSON‑RPC‑style endpoint that the LLM can call
3. **Scraping API** – provides reliable access to public web pages with automatic header rotation, JavaScript rendering, and CAPTCHA bypass

The LLM never sees the raw HTML; it receives a cleaned JSON payload (or markdown/text) that it can immediately use in its reasoning loop.

1. **LLM decides to scrape** — 
2. **MCP server receives the call** — 
3. **API returns structured data** — 
4. **LLM consumes the result** — 
5. **Optional feedback loop** — 

## Setting Up the MCP Server
You can run the MCP server locally or in the cloud. Below is a minimal Python implementation using FastAPI. It exposes a single `/tool/scrape` endpoint that forwards the request to AlterLab.

```python title="mcp_server.py" {7-12}
from fastapi import FastAPI, HTTPException
import httpx
import os

app = FastAPI()
ALTERLAB_KEY = os.getenv("ALTERLAB_API_KEY")
ALTERLAB_ENDPOINT = "https://api.alterlab.io/v1/scrape"

@app.post("/tool/scrape")
async def scrape_tool(payload: dict):
    url = payload.get("url")
    if not url:
        raise HTTPException(status_code=400, detail="Missing 'url' field")
    async with httpx.AsyncClient() as client:
        response = await client.post(
            ALTERLAB_ENDPOINT,
            json={"url": url, "formats": ["json"]},
            headers={"X-API-Key": ALTERLAB_KEY},
            timeout=30.0,
        )
        if response.status_code != 200:
            raise HTTPException(status_code=502, detail="Scraping failed")
        return response.json()
```

The server expects a JSON body with a `url` field and returns the raw JSON from AlterLab. You can extend it to accept additional parameters like `min_tier` or `formats`.

## Integrating with LLM Tool Use
Most LLM frameworks (e.g., LangChain, LlamaIndex, or custom agents) let you define tools as functions that return a string or structured object. You point the tool at your MCP server’s endpoint. Below is an example using a pseudo‑agent loop; replace the HTTP call with your framework’s tool invocation method.

```python title="agent_loop.py" {5-10}
import httpx
import json

MCP_ENDPOINT = "http://localhost:8000/tool/scrape"

def scrape_tool(url: str) -> dict:
    resp = httpx.post(MCP_ENDPOINT, json={"url": url})
    resp.raise_for_status()
    return resp.json()

# Example agent reasoning step
user_query = "What is the latest headline on the tech news site?"
if "latest" in user_query.lower():
    # LLM decides to scrape
    data = scrape_tool("https://example-news-site.com/latest")
    headline = data.get("title", "No title found")
    print(f"Fetched headline: {headline}")
else:
    print("Answering from internal knowledge")
```

The agent checks the user intent, calls the scraping tool when fresh data is needed, and incorporates the result into its response.

## Code Examples: Python SDK and cURL
You can also call AlterLab directly from your agent if you prefer not to run an MCP server. The SDK handles retries, header rotation, and response parsing.

```python title="direct_sdk.py" {3-6}
import alterlab

client = alterlab.Client(os.getenv("ALTERLAB_API_KEY"))
response = client.scrape(
    "https://example-news-site.com/latest",
    formats=["json"],
    # Optionally force a higher tier for Java

## Frequently Asked Questions

### What is an agentic web browsing tool?

An agentic web browsing tool uses LLMs to decide when and how to fetch live web data through APIs, enabling autonomous actions based on current information.

### How do MCP servers fit into LLM tool use?

MCP servers expose functions (like web scraping) as tools that LLMs can call via a standardized interface, letting the model retrieve real-time data without writing custom integration code.

### Do I need to handle anti-bot measures myself when using this approach?

No—the scraping API handles proxy rotation, JavaScript rendering, and bot detection bypass, so your agent receives clean data from public pages.

## Related

- [Reduce LLM Token Waste in RAG with Structured Markdown and JSON Extraction](<https://alterlab.io/blog/reduce-llm-token-waste-in-rag-with-structured-markdown-and-json-extraction>)
- [Scaling Web Scraping Pipelines for Production Data](<https://alterlab.io/blog/scaling-web-scraping-pipelines-for-production-data>)
- [How to Scrape Niche.com Data: Complete Guide for 2026](<https://alterlab.io/blog/how-to-scrape-niche-com-data-complete-guide-for-2026>)