Intermediate4 steps

How to Scrape Product Reviews

Product reviews are spread across paginated review sections, often loaded lazily or hidden behind "Show more" interactions. Collecting a complete review dataset requires handling JavaScript rendering, pagination, and possibly scroll-triggered loading.

Step-by-Step Guide

1

Find the review section URL pattern

Many sites load reviews on a separate paginated URL (e.g., /product/123/reviews?page=2) or via a dedicated API endpoint visible in browser DevTools Network tab.

2

Fetch reviews with JavaScript rendering

Send the review page URL to AlterLab with render_js enabled to ensure lazy-loaded review content is included in the response.

3

Extract review metadata

Parse each review for: rating (numeric), review text, reviewer name, date, and verified purchase status when available.

4

Paginate through all reviews

Follow the review pagination to collect all available reviews. Sort by 'most recent' to prioritize fresh data.

Code Example

Python
import requests
from bs4 import BeautifulSoup

def scrape_reviews(product_url: str, api_key: str) -> list[dict]:
    response = requests.post(
        "https://alterlab.io/api/v1/scrape",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json={"url": product_url, "render_js": True},
    )
    html = response.json().get("html", "")
    soup = BeautifulSoup(html, "html.parser")

    reviews = []
    for review in soup.select(".review-item"):
        reviews.append({
            "rating": review.select_one("[data-rating]")?.get("data-rating"),
            "text": review.select_one(".review-body")?.get_text(strip=True),
            "author": review.select_one(".reviewer-name")?.get_text(strip=True),
            "date": review.select_one("time")?.get("datetime"),
        })
    return reviews

Replace YOUR_API_KEY with your key from the . No credit card required.

Try this yourself with AlterLab

Run this tutorial on live websites with AlterLab's API. Free tier includes 5,000 requests — no credit card required.

View API docs

Frequently Asked Questions

Can I extract star ratings as numeric values?

Ratings are often embedded in data attributes (data-rating='4.5') or in aria-label text ('4.5 out of 5 stars'). Check both locations when writing your extractor.

Responsible Use

AlterLab is designed for extracting publicly available data. Always review the terms of service for any website you access, respect robots.txt directives, and ensure your use case complies with applicable laws in your jurisdiction.

More tutorials

Browse all how-to guides for web scraping — from beginner extractions to advanced multi-page pipelines.

Your first scrape.
Sixty seconds.

$1 free credit — up to 5,000 scrapes. No credit card.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · $1 free credit, up to 5,000 scrapes · Balance never expires