Semantic Search with Embeddings 2026 — Complete Python Guide

Introduction

Why This Matters

Traditional keyword search fails when users use synonyms, paraphrases, or intent-based queries. "How do I cancel my subscription?" does not match "subscription cancellation process" without semantic understanding. Semantic search closes this gap by comparing the meaning of queries and documents, not their exact words.

In 2026, semantic search is a core feature in support portals, documentation sites, e-commerce product search, and internal knowledge bases. It is typically 60-80% more effective than keyword search for natural language queries while requiring no query reformulation from the user.

The technology is simple: convert text to dense vector representations using an embedding model, store those vectors, then find the most similar vectors to any new query at search time. The hard parts are chunking strategy, hybrid search, and cost optimization at scale.

How Embeddings Work

from openai import OpenAI
import numpy as np
 
client = OpenAI()
 
def embed(text: str) -> list[float]:
    return client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    ).data[0].embedding
 
vec1 = embed("Python programming language")
vec2 = embed("coding in Python")
vec3 = embed("recipe for chocolate cake")
 
def cosine_similarity(a, b):
    a, b = np.array(a), np.array(b)
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
 
print(cosine_similarity(vec1, vec2))  # ~0.92 — semantically similar
print(cosine_similarity(vec1, vec3))  # ~0.12 — semantically different

Build a Document Search Engine

import numpy as np
import json
from pathlib import Path
 
class SemanticSearchEngine:
    def __init__(self, model: str = "text-embedding-3-small"):
        self.model = model
        self.documents: list[str] = []
        self.embeddings: np.ndarray | None = None
        self.metadata: list[dict] = []
 
    def embed_batch(self, texts: list[str]) -> np.ndarray:
        response = client.embeddings.create(model=self.model, input=texts)
        return np.array([item.embedding for item in response.data])
 
    def index(self, documents: list[str], metadata: list[dict] | None = None):
        self.documents = documents
        self.metadata = metadata or [{} for _ in documents]
        all_embeddings = []
        for i in range(0, len(documents), 100):
            batch = documents[i:i+100]
            all_embeddings.append(self.embed_batch(batch))
            print(f"Indexed {min(i+100, len(documents))}/{len(documents)}")
        self.embeddings = np.vstack(all_embeddings)
 
    def search(self, query: str, top_k: int = 5) -> list[dict]:
        query_embedding = self.embed_batch([query])[0]
        norms = np.linalg.norm(self.embeddings, axis=1) * np.linalg.norm(query_embedding)
        similarities = self.embeddings @ query_embedding / norms
        top_indices = np.argsort(similarities)[::-1][:top_k]
        return [
            {
                "document": self.documents[i],
                "score": float(similarities[i]),
                "metadata": self.metadata[i],
            }
            for i in top_indices
        ]
 
    def save(self, path: str):
        Path(path).mkdir(exist_ok=True)
        np.save(f"{path}/embeddings.npy", self.embeddings)
        with open(f"{path}/data.json", "w") as f:
            json.dump({"documents": self.documents, "metadata": self.metadata}, f)
 
    def load(self, path: str):
        self.embeddings = np.load(f"{path}/embeddings.npy")
        with open(f"{path}/data.json") as f:
            data = json.load(f)
        self.documents = data["documents"]
        self.metadata = data["metadata"]
 
# Usage
engine = SemanticSearchEngine()
engine.index(
    documents=["Python is a high-level programming language", "React is a JavaScript UI library"],
    metadata=[{"topic": "python"}, {"topic": "react"}]
)
results = engine.search("how to build web apps", top_k=3)
for r in results:
    print(f"Score: {r['score']:.3f} | {r['document'][:60]}")

Hybrid Search: Vector + BM25

Pure semantic search misses exact keyword matches. Hybrid search combines vector similarity with BM25 keyword scoring:

from rank_bm25 import BM25Okapi
 
class HybridSearchEngine(SemanticSearchEngine):
    def __init__(self, *args, alpha: float = 0.5, **kwargs):
        super().__init__(*args, **kwargs)
        self.bm25 = None
        self.alpha = alpha  # 1.0 = pure semantic, 0.0 = pure BM25
 
    def index(self, documents: list[str], metadata=None):
        super().index(documents, metadata)
        tokenized = [doc.lower().split() for doc in documents]
        self.bm25 = BM25Okapi(tokenized)
 
    def search(self, query: str, top_k: int = 5) -> list[dict]:
        semantic_results = super().search(query, top_k=len(self.documents))
        semantic_scores = {r["document"]: r["score"] for r in semantic_results}
 
        bm25_scores = self.bm25.get_scores(query.lower().split())
        bm25_max = max(bm25_scores) if max(bm25_scores) > 0 else 1
 
        combined = []
        for i, doc in enumerate(self.documents):
            sem_score = semantic_scores.get(doc, 0)
            bm25_score = bm25_scores[i] / bm25_max
            combined_score = self.alpha * sem_score + (1 - self.alpha) * bm25_score
            combined.append((combined_score, i))
 
        combined.sort(reverse=True)
        return [
            {"document": self.documents[i], "score": score, "metadata": self.metadata[i]}
            for score, i in combined[:top_k]
        ]

FastAPI Search Service

from fastapi import FastAPI
from pydantic import BaseModel
 
app = FastAPI()
engine = HybridSearchEngine()
engine.load("./search_index")
 
class SearchRequest(BaseModel):
    query: str
    top_k: int = 5
    alpha: float = 0.5
 
@app.post("/search")
async def search(req: SearchRequest):
    engine.alpha = req.alpha
    results = engine.search(req.query, req.top_k)
    return {"query": req.query, "results": results}
 
@app.post("/index")
async def index(documents: list[str]):
    engine.index(documents)
    engine.save("./search_index")
    return {"indexed": len(documents)}

Cost Optimization

# text-embedding-3-small: $0.02/1M tokens (vs $0.13 for large)
# Use small for bulk indexing, large for precision-critical search
 
import hashlib
 
embedding_cache = {}
 
def cached_embed(text: str) -> list[float]:
    key = hashlib.md5(text.encode()).hexdigest()
    if key not in embedding_cache:
        embedding_cache[key] = embed(text)
    return embedding_cache[key]

Common Mistakes / Pitfalls

Embedding entire documents — always chunk into 500-1000 token pieces before embedding
Using cosine similarity with unnormalized vectors — always normalize before computing dot products
Re-embedding the same content repeatedly — cache embeddings by content hash; text does not change
Ignoring model dimensions — text-embedding-3-small (1536 dims) vs large (3072 dims) affects both quality and storage cost
No metadata filtering — results without category/date filters are less useful; always store metadata with vectors

Best Practices

Use text-embedding-3-small for cost efficiency and text-embedding-3-large when precision is critical
Batch embed in groups of 100 — the API batches efficiently and avoids rate limits
Save embeddings to disk with numpy — avoids re-computing on restart and keeps startup fast
Set alpha=0.7 for semantic-heavy queries (questions, concepts) and 0.3 for keyword-heavy queries (names, codes)
Run hybrid search with alpha as a tunable parameter that users can adjust via UI slider

Key Takeaways

Semantic search converts text to dense vectors and finds similar vectors at query time via cosine similarity
OpenAI text-embedding-3-small costs $0.02 per million tokens — affordable even for large corpora
Hybrid search combining vector embeddings (alpha) and BM25 keyword scoring outperforms either alone
Embedding caching by content hash eliminates redundant API calls when documents do not change
The numpy dot-product approach scales to millions of documents before requiring a dedicated vector database
Chunking documents into 500-1000 token pieces before embedding improves retrieval precision
FastAPI makes it trivial to expose the search engine as a REST service with a typed Pydantic schema
alpha=0.5 (equal weight) is a safe default; tune based on your query distribution in production

Introduction#

Why This Matters#

How Embeddings Work#

Build a Document Search Engine#

Hybrid Search: Vector + BM25#

FastAPI Search Service#

Cost Optimization#

Common Mistakes / Pitfalls#

Best Practices#

Key Takeaways#

Related reading