ai-agent

Vector Store

A vector store is a database optimised for storing and querying high-dimensional embedding vectors by semantic similarity, enabling fast nearest-neighbour search at scale.

Traditional databases match rows by exact value equality or range conditions on scalar fields. Vector stores instead index floating-point vectors and answer queries of the form 'find the K rows whose vectors are most similar to this query vector'. The similarity metric is typically cosine similarity or dot product, implemented with approximate nearest-neighbour (ANN) algorithms such as HNSW or IVF for performance at millions or billions of vectors.

Popular vector stores include Pinecone, Weaviate, Qdrant, Chroma (embedded), pgvector (PostgreSQL extension), and Milvus. Most support hybrid search — combining vector similarity with traditional metadata filters — so a RAG pipeline can, for example, retrieve the most semantically relevant chunks that were also scraped within the last 24 hours.

For scraping pipelines, the vector store sits at the end of the ingestion chain: scraped content → chunk → embed → upsert. At query time, a user's question is embedded and the ANN index returns the most relevant chunks, which are then sent to the LLM for answer generation.

Examples

# Qdrant: upsert scraped content and search
from qdrant_client import QdrantClient
from qdrant_client.models import PointStruct

client = QdrantClient(url="http://localhost:6333")
client.upsert("web_content", points=[
    PointStruct(id=1, vector=embed("Scraped text here"),
                payload={"url": "https://example.com", "date": "2026-06-25"})
])
results = client.search("web_content", query_vector=embed("my question"), limit=5)

Related Terms

Embeddings
Embeddings are dense vector representations of text (or other data) produced by a neural network, where semantically similar content maps to nearby points in vector space.
RAG (Retrieval-Augmented Generation)
RAG is an AI architecture that grounds LLM responses in retrieved documents, combining a vector search step with language model generation to reduce hallucinations.
LLM (Large Language Model)
A Large Language Model is a neural network trained on vast text corpora that can generate, summarise, translate, and reason over natural language at human level.
Data Extraction
The process of identifying and pulling specific fields from raw web content, converting unstructured HTML into typed JSON records.
Webhook
An HTTP callback that delivers data to a specified URL when an async scrape job completes.

Extract Vector Store data from any website

AlterLab returns clean, structured data from any public URL — no scraper infrastructure needed. Start free, no credit card required.

View API docs
Back to Glossary

Your first scrape.
Sixty seconds.

$1 free balance. No credit card. No SDK.Just a POST request.

terminal
curl -X POST https://api.alterlab.io/v1/scrape \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'

No credit card required · Up to 5,000 free scrapes · Balance never expires

    Vector Store — Web Scraping Glossary | AlterLab