```yaml
product: AlterLab
title: "How to Build a Web Scraper API Endpoint | AlterLab | AlterLab"
comparison_context: "AlterLab is an alternative to Firecrawl, ScrapingBee, and Bright Data."
source_url: https://alterlab.io/how-to/build-web-scraper-api
```

Advanced4 steps

# How to Build a Web Scraper API Endpoint

If multiple services or team members need access to scraped data, a dedicated scraper API endpoint is more efficient than running scrapers in each service independently. A single API layer handles the scraping, caching, and data normalization.

## Step-by-Step Guide

1

### Set up a FastAPI project

Install FastAPI and uvicorn: `pip install fastapi uvicorn`. Create a main.py with a FastAPI app instance.

2

### Create a scrape endpoint

Define a POST endpoint that accepts a URL and optional parameters, then calls the AlterLab API internally and returns parsed data.

3

### Add response caching

Cache scraping results in Redis or a simple in-memory dict with a TTL to avoid re-scraping the same URL repeatedly within a short window.

4

### Add error handling

Return structured error responses when the AlterLab API returns a non-200 status or when HTML parsing fails to find expected elements.

## Code Example

Python

```
from fastapi import FastAPI, HTTPException
import requests

app = FastAPI()
API_KEY = "YOUR_API_KEY"

@app.post("/scrape")
async def scrape(url: str, selector: str):
    response = requests.post(
        "https://alterlab.io/api/v1/scrape",
        headers={"X-API-Key": API_KEY, "Content-Type": "application/json"},
        json={"url": url, "render_js": "auto"},
    )
    if response.status_code != 200:
        raise HTTPException(status_code=502, detail="Scraping failed")

    from bs4 import BeautifulSoup
    html = response.json().get("html", "")
    soup = BeautifulSoup(html, "html.parser")
    elements = [el.get_text(strip=True) for el in soup.select(selector)]

    return {"url": url, "selector": selector, "results": elements}
```

Replace `YOUR_API_KEY` with your key from the . No credit card required.

## Try this yourself with AlterLab

Run this tutorial on live websites with AlterLab's API. New accounts get up to 5,000 free requests — no credit card required.

[View API docs](https://alterlab.io/docs/api/rest)

## Frequently Asked Questions

### Should I cache scraping results?

Yes — caching is important for cost control and latency. Cache responses with a TTL that matches how frequently the data changes: minutes for live pricing, hours for product details, days for static content.

### Responsible Use

AlterLab is designed for extracting publicly available data. Always review the terms of service for any website you access, respect `robots.txt` directives, and ensure your use case complies with applicable laws in your jurisdiction.

### More tutorials

Browse all [how-to guides](https://alterlab.io/how-to) for web scraping — from beginner extractions to advanced multi-page pipelines.

## Your first scrape. Sixty seconds.

Up to 5,000 free requests — about 5,000 Light, 500 Protected or 250 Dynamic requests. No credit card. 
Just a POST request.

terminal

curl -X POST https://api.alterlab.io/v1/scrape \

-H "X-API-Key: YOUR_KEY" \

-H "Content-Type: application/json" \

-d '{"url": "https://example.com", "formats": ["markdown"]}'

[Start building free](https://alterlab.io/signin)

No credit card required · Up to 5,000 free requests (about 5,000 Light, 500 Protected or 250 Dynamic requests) · Balance never expires