general

Lazy Loading

Lazy loading defers fetching images or content until the user scrolls them into view, reducing initial page load time but requiring scrapers to trigger scroll events to access deferred content.

Browsers support native lazy loading via the `loading='lazy'` attribute on `<img>` and `<iframe>` elements, which defers their fetch until they approach the viewport. JavaScript frameworks also implement lazy loading for arbitrary content using IntersectionObserver: when a placeholder element scrolls into view, JavaScript replaces it with the actual content.

For HTML-only scrapers, lazy-loaded images appear in the raw HTML as empty `<img>` tags with their actual URL in a `data-src` attribute rather than `src`. The actual image is only fetched when `src` is populated by JavaScript after scroll. Extracting the real URL requires either executing the page's JavaScript or reading the `data-src` attribute directly.

For lazy-loaded text content (common in social feeds and virtual-scroll lists), a headless browser is required to simulate scrolling and wait for the IntersectionObserver to populate the content. See also: Infinite Scroll.

Examples

# Extract lazy-loaded image URLs from data-src
from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
images = []
for img in soup.find_all("img"):
    src = img.get("data-src") or img.get("src") or img.get("data-lazy-src")
    if src:
        images.append(src)