Intermediate4 steps

How to Scrape Job Listings

Job boards render listings dynamically, apply geographic and device-based filtering, and paginate results across hundreds of pages. Building a job data pipeline requires handling JavaScript rendering and systematic pagination.

Step-by-Step Guide

1

Construct your job search URL

Most job boards accept search parameters in the URL — job title, location, and date posted. Build the search URL that returns the listings you need.

2

Fetch with JavaScript rendering

Send the search URL to AlterLab with render_js enabled. Job boards frequently lazy-load listings and filter counts via JavaScript.

3

Extract listing data from result cards

Parse job title, company name, location, salary range (when available), and the job detail page URL from each listing card.

4

Paginate through all results

Follow the next-page links to collect all listings. Many job boards limit pagination to 20–40 pages — start with the most recent listings first.

Code Example

Python
import requests
from bs4 import BeautifulSoup

def scrape_jobs(search_url: str, api_key: str) -> list[dict]:
    response = requests.post(
        "https://alterlab.io/api/v1/scrape",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json={"url": search_url, "render_js": True},
    )
    html = response.json().get("html", "")
    soup = BeautifulSoup(html, "html.parser")

    jobs = []
    for card in soup.select(".job-card"):
        jobs.append({
            "title": card.select_one(".job-title")?.get_text(strip=True),
            "company": card.select_one(".company-name")?.get_text(strip=True),
            "location": card.select_one(".location")?.get_text(strip=True),
            "url": card.select_one("a")?.get("href"),
        })
    return jobs

Replace YOUR_API_KEY with your key from the . No credit card required.

Try this yourself with AlterLab

Run this tutorial on live websites with AlterLab's API. Free tier includes 5,000 requests — no credit card required.

View API docs

Frequently Asked Questions

Can I scrape job boards that require login?

Many job boards display a subset of listings without login. For boards that require authentication, you will need a valid account and session cookie management, which is outside the scope of standard scraping.

Responsible Use

AlterLab is designed for extracting publicly available data. Always review the terms of service for any website you access, respect robots.txt directives, and ensure your use case complies with applicable laws in your jurisdiction.

More tutorials

Browse all how-to guides for web scraping — from beginner extractions to advanced multi-page pipelines.

Your first scrape.
Sixty seconds.

$1 free credit — up to 5,000 scrapes. No credit card.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · $1 free credit, up to 5,000 scrapes · Balance never expires