Building an LLM Agent That Writes a Daily Social Feed Digest

Short answer

How can an LLM agent browse a public social feed and write a digest?

Give the agent one tool that fetches a public page and returns clean markdown, then have it summarise the posts into themes, links and notable replies. Keep fetching outside the model: the tool handles rendering and retries, the model only reads text. AlterLab's API and MCP server provide that fetch tool for agent frameworks.

Step by step

  1. 1

    Define a fetch tool

    Expose a single function such as fetch_page(url) that returns markdown. Agents use simple, well-described tools more reliably than raw HTML.

  2. 2

    Fetch public pages only

    Point the tool at publicly available profile or list pages, and respect each platform's terms of service and robots.txt.

  3. 3

    Summarise in a fixed format

    Ask the model for a digest with sections such as top themes, notable posts with links and open questions, so day-to-day output stays comparable.

  4. 4

    Schedule the run

    Run the agent once a day from a scheduler and store the digest, so the model never needs to remember previous days.

Example

Python
import requests

def fetch_page(url: str) -> str:
    """Tool for the agent: return a public page as markdown."""
    response = requests.post(
        "https://alterlab.io/api/v1/scrape",
        headers={"X-API-Key": "YOUR_API_KEY", "Content-Type": "application/json"},
        json={"url": url, "render_js": "auto", "formats": ["markdown"]},
    )
    return response.json().get("content", {}).get("markdown", "")

Frequently Asked Questions

Why markdown instead of HTML for the agent?

Markdown keeps headings, links and lists but drops scripts and styling, so the model spends its context on the posts themselves and the digest costs fewer tokens.

Can I plug this into an MCP client?

Yes. AlterLab ships an MCP server, so Claude, Cursor and other MCP clients can call the fetch tool directly without custom glue code.

Read the full guide

Web data for AI agents

Try it with AlterLab

Send a URL, get the page back as HTML, markdown or JSON. Rendering, proxy rotation and retries are handled for you.

View API docs

Your first scrape.
Sixty seconds.

Up to 5,000 free requests — about 5,000 Light, 500 Protected or 250 Dynamic requests. No credit card.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · Up to 5,000 free requests (about 5,000 Light, 500 Protected or 250 Dynamic requests) · Balance never expires