How to Monitor Competitor Websites
Staying aware of competitor pricing, product launches, and messaging changes requires checking multiple websites repeatedly. Manual monitoring doesn't scale — you need an automated pipeline that tracks changes and sends alerts.
Step-by-Step Guide
Define what to monitor
List the specific pages and data points to track: pricing pages, product catalogs, homepage messaging, job listings. Focused monitoring reduces noise and cost.
Establish a baseline snapshot
Fetch each page and store the current state of the data points you care about. This becomes your reference for detecting future changes.
Schedule recurring checks
Run your monitoring script on a schedule — hourly for pricing, daily for content. Compare each new fetch against your stored baseline.
Detect and classify changes
Compare new data against the baseline. Classify changes by type (price drop, new feature, messaging change) and severity.
Send alerts
Route change notifications to your preferred channel — Slack webhook, email, or your internal dashboard.
Code Example
import requests
import json
from pathlib import Path
def check_competitor(url: str, selector: str, api_key: str, state_file: str) -> dict:
response = requests.post(
"https://alterlab.io/api/v1/scrape",
headers={"X-API-Key": api_key, "Content-Type": "application/json"},
json={"url": url},
)
from bs4 import BeautifulSoup
html = response.json().get("html", "")
current = BeautifulSoup(html, "html.parser").select_one(selector)
current_text = current.get_text(strip=True) if current else ""
path = Path(state_file)
previous = json.loads(path.read_text()).get("value") if path.exists() else None
path.write_text(json.dumps({"value": current_text}))
return {"changed": current_text != previous, "previous": previous, "current": current_text}Replace YOUR_API_KEY with your key from the . No credit card required.
Try this yourself with AlterLab
Run this tutorial on live websites with AlterLab's API. Free tier includes 5,000 requests — no credit card required.
Frequently Asked Questions
How frequently should I check competitor sites?
Check pricing pages every 1–4 hours, product catalogs daily, and content/blog pages weekly. More frequent checks increase costs and place more load on the target site.
Responsible Use
AlterLab is designed for extracting publicly available data. Always review the terms of service for any website you access, respect robots.txt directives, and ensure your use case complies with applicable laws in your jurisdiction.
More tutorials
Browse all how-to guides for web scraping — from beginner extractions to advanced multi-page pipelines.
Your first scrape.
Sixty seconds.
$1 free credit — up to 5,000 scrapes. No credit card.
Just a POST request.
No credit card required · $1 free credit, up to 5,000 scrapes · Balance never expires