Building an LLM Agent That Writes a Daily Social Feed Digest
Short answer
How can an LLM agent browse a public social feed and write a digest?
Give the agent one tool that fetches a public page and returns clean markdown, then have it summarise the posts into themes, links and notable replies. Keep fetching outside the model: the tool handles rendering and retries, the model only reads text. AlterLab's API and MCP server provide that fetch tool for agent frameworks.
Step by step
- 1
Define a fetch tool
Expose a single function such as fetch_page(url) that returns markdown. Agents use simple, well-described tools more reliably than raw HTML.
- 2
Fetch public pages only
Point the tool at publicly available profile or list pages, and respect each platform's terms of service and robots.txt.
- 3
Summarise in a fixed format
Ask the model for a digest with sections such as top themes, notable posts with links and open questions, so day-to-day output stays comparable.
- 4
Schedule the run
Run the agent once a day from a scheduler and store the digest, so the model never needs to remember previous days.
Example
import requests
def fetch_page(url: str) -> str:
"""Tool for the agent: return a public page as markdown."""
response = requests.post(
"https://alterlab.io/api/v1/scrape",
headers={"X-API-Key": "YOUR_API_KEY", "Content-Type": "application/json"},
json={"url": url, "render_js": "auto", "formats": ["markdown"]},
)
return response.json().get("content", {}).get("markdown", "")Frequently Asked Questions
Why markdown instead of HTML for the agent?
Markdown keeps headings, links and lists but drops scripts and styling, so the model spends its context on the posts themselves and the digest costs fewer tokens.
Can I plug this into an MCP client?
Yes. AlterLab ships an MCP server, so Claude, Cursor and other MCP clients can call the fetch tool directly without custom glue code.
Read the full guide
Web data for AI agentsTry it with AlterLab
Send a URL, get the page back as HTML, markdown or JSON. Rendering, proxy rotation and retries are handled for you.
Your first scrape.
Sixty seconds.
Up to 5,000 free requests — about 5,000 Light, 500 Protected or 250 Dynamic requests. No credit card.
Just a POST request.
No credit card required · Up to 5,000 free requests (about 5,000 Light, 500 Protected or 250 Dynamic requests) · Balance never expires