ai-agent

RAG (Retrieval-Augmented Generation)

RAG is an AI architecture that grounds LLM responses in retrieved documents, combining a vector search step with language model generation to reduce hallucinations.

Retrieval-Augmented Generation (RAG) addresses the core limitation of large language models: their knowledge is frozen at training time and they cannot access live information. RAG augments the generation step by first querying an external knowledge base — typically a vector database indexed with embeddings — for documents relevant to the user's question. Those documents are injected into the LLM's context window before generation, giving the model fresh, citable grounding.

The pipeline has three main components: a retrieval system (embedding model + vector store + similarity search), a context assembler that selects and formats the retrieved chunks, and an LLM that generates the final answer conditioned on both the query and the retrieved context. RAG is widely used in enterprise chatbots, document Q&A systems, and AI search interfaces.

Web scraping feeds RAG pipelines: scraped content is chunked, embedded, and indexed so that AI agents can query live web data rather than relying on stale training knowledge. AlterLab's structured extraction output is well-suited as a RAG ingestion source.

Examples

# LangChain RAG pipeline sketch
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA

vectordb = Chroma(embedding_function=OpenAIEmbeddings())
retriever = vectordb.as_retriever(search_kwargs={"k": 5})
chain = RetrievalQA.from_chain_type(ChatOpenAI(), retriever=retriever)
answer = chain.run("What is the current price of product X?")

Related Terms

Embeddings
Embeddings are dense vector representations of text (or other data) produced by a neural network, where semantically similar content maps to nearby points in vector space.
Vector Store
A vector store is a database optimised for storing and querying high-dimensional embedding vectors by semantic similarity, enabling fast nearest-neighbour search at scale.
LLM (Large Language Model)
A Large Language Model is a neural network trained on vast text corpora that can generate, summarise, translate, and reason over natural language at human level.
MCP (Model Context Protocol)
MCP is an open protocol by Anthropic that standardises how AI models connect to external tools, APIs, and data sources through a uniform server-client interface.
Data Extraction
The process of identifying and pulling specific fields from raw web content, converting unstructured HTML into typed JSON records.

Extract RAG (Retrieval-Augmented Generation) data from any website

AlterLab returns clean, structured data from any public URL — no scraper infrastructure needed. Start free, no credit card required.

View API docs
Back to Glossary

Your first scrape.
Sixty seconds.

$1 free balance. No credit card. No SDK.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · Up to 5,000 free scrapes · Balance never expires

    RAG (Retrieval-Augmented Generation) — Web Scraping Glossary | AlterLab