Beginner4 steps

How to Scrape Data with Python

Python is the most popular language for web scraping — with excellent libraries for HTTP requests, HTML parsing, and data processing. This guide covers the complete setup from API key to clean extracted data.

Step-by-Step Guide

1

Install required libraries

Install the requests and beautifulsoup4 libraries: `pip install requests beautifulsoup4`. These handle HTTP requests and HTML parsing respectively.

2

Fetch a page with AlterLab

Use the requests library to POST to the AlterLab API with your target URL and API key. The response contains the rendered HTML.

3

Parse with BeautifulSoup

Pass the HTML to BeautifulSoup for parsing. Use CSS selectors (`soup.select()`) or tag finders (`soup.find()`) to extract the data you need.

4

Store the extracted data

Write your results to a CSV with the csv module, a JSON file, or a database. Use pandas for larger datasets that benefit from tabular operations.

Code Example

PythonGet free API key →
import requests
from bs4 import BeautifulSoup
import csv

API_KEY = "YOUR_API_KEY"

def scrape_page(url: str) -> list[dict]:
    response = requests.post(
        "https://alterlab.io/api/v1/scrape",
        headers={"X-API-Key": API_KEY, "Content-Type": "application/json"},
        json={"url": url, "render_js": True},
    )
    html = response.json().get("html", "")
    soup = BeautifulSoup(html, "html.parser")

    return [
        {"title": el.get_text(strip=True)}
        for el in soup.select("h2.article-title")
    ]

results = scrape_page("https://example.com/articles")

with open("output.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["title"])
    writer.writeheader()
    writer.writerows(results)

Replace YOUR_API_KEY with your key from the dashboard. No credit card required.

Ready to try it?

Run this tutorial on live websites with AlterLab's API. Start free — no credit card required.

Get free API keyView API docs

Frequently Asked Questions

Responsible Use

AlterLab is designed for extracting publicly available data. Always review the terms of service for any website you access, respect robots.txt directives, and ensure your use case complies with applicable laws in your jurisdiction.

More tutorials

Browse all how-to guides for web scraping — from beginner extractions to advanced multi-page pipelines.

Your first scrape.
Sixty seconds.

$1 free balance. No credit card. No SDK.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · Up to 5,000 free scrapes · Balance never expire

    How to Scrape Data with Python — Complete Guide | AlterLab | AlterLab