OpenAI Embeddings API — Complete Tutorial for Semantic Search and RAG

Introduction

Why This Matters

OpenAI's embedding models are the most widely used text embeddings in production AI applications. The text-embedding-3-small and text-embedding-3-large models released in January 2024 significantly outperformed the previous ada-002 model on the MTEB benchmark while reducing cost by up to 5x. For any team building RAG systems, semantic search, or classification pipelines in Python, the OpenAI Embeddings API is the natural starting point.

Understanding which model to choose, how to batch calls efficiently, how to reduce dimensionality without losing quality, and how to build a complete retrieval pipeline saves significant time and API cost in production.

API Setup

pip install openai numpy python-dotenv

import os
from openai import OpenAI
from dotenv import load_dotenv
 
load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

Available Models

OpenAI offers three embedding models as of 2025:

Model	Dimensions	MTEB Score	Cost	Best For
text-embedding-3-large	3072	64.6	$0.13/1M tokens	Highest quality, research
text-embedding-3-small	1536	62.3	$0.02/1M tokens	Production default
text-embedding-ada-002	1536	61.0	$0.10/1M tokens	Legacy — avoid for new projects

The text-embedding-3-small model is the recommended default: it provides 62% better performance than ada-002 on MTEB at 20% of the price.

Basic Embedding Call

from openai import OpenAI
import numpy as np
 
client = OpenAI()
 
# Single text
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="The quick brown fox jumps over the lazy dog."
)
 
embedding = response.data[0].embedding
print(f"Dimension: {len(embedding)}")         # 1536
print(f"Type: {type(embedding[0])}")          # float
print(f"Tokens used: {response.usage.total_tokens}")

Batch Embedding (Critical for Cost Efficiency)

Always batch multiple texts in a single API call — OpenAI processes them together at the same cost per token:

from openai import OpenAI
import numpy as np
 
client = OpenAI()
 
def embed_batch(texts: list[str], model: str = "text-embedding-3-small") -> list[np.ndarray]:
    """Embed a batch of texts in a single API call."""
    # OpenAI recommends replacing newlines for better performance
    texts = [text.replace("\n", " ") for text in texts]
 
    response = client.embeddings.create(
        model=model,
        input=texts
    )
 
    # Sort by index to ensure order matches input
    embeddings = sorted(response.data, key=lambda x: x.index)
    return [np.array(item.embedding) for item in embeddings]
 
documents = [
    "Our refund policy allows returns within 30 days.",
    "Enterprise plans include 99.9% SLA.",
    "API rate limits are 1000 requests per minute.",
    "Customer support is available 24/7 for paid plans.",
]
 
embeddings = embed_batch(documents)
print(f"Embedded {len(embeddings)} documents, {len(embeddings[0])} dimensions each")

For large datasets, batch in groups of 2048 texts (API limit):

def embed_large_dataset(texts: list[str], batch_size: int = 512) -> list[np.ndarray]:
    all_embeddings = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        batch_embeddings = embed_batch(batch)
        all_embeddings.extend(batch_embeddings)
        print(f"Embedded {min(i + batch_size, len(texts))}/{len(texts)} texts")
    return all_embeddings

Dimensionality Reduction

OpenAI's v3 models support native dimensionality reduction — shorter vectors with minimal quality loss:

# Standard 1536 dimensions
full_response = client.embeddings.create(
    model="text-embedding-3-small",
    input="semantic search example"
)
print(len(full_response.data[0].embedding))  # 1536
 
# Reduced to 256 dimensions (75% memory savings, ~2% quality loss)
reduced_response = client.embeddings.create(
    model="text-embedding-3-small",
    input="semantic search example",
    dimensions=256
)
print(len(reduced_response.data[0].embedding))  # 256

When to use reduced dimensions:

Storing millions of vectors — 256 dims vs 1536 saves 83% storage and memory
Latency-sensitive applications — smaller vectors mean faster ANN search
Cost-sensitive production — smaller payloads reduce transfer costs

Cosine Similarity and Search

import numpy as np
from openai import OpenAI
 
client = OpenAI()
 
def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
 
# Index documents
documents = [
    "How to cancel a subscription in account settings.",
    "Refund requests must be submitted within 30 days.",
    "Enterprise contracts include custom SLA terms.",
    "The API accepts JSON payloads via POST requests.",
]
 
doc_embeddings = embed_batch(documents)
 
# Query
query = "I want to stop my plan"
query_embedding = embed_batch([query])[0]
 
# Rank by similarity
scores = [(doc, cosine_similarity(query_embedding, doc_emb))
          for doc, doc_emb in zip(documents, doc_embeddings)]
scores.sort(key=lambda x: x[1], reverse=True)
 
print("Results:")
for doc, score in scores:
    print(f"  {score:.4f} | {doc}")

Full RAG Pipeline

from openai import OpenAI
import numpy as np
import json
 
client = OpenAI()
 
class OpenAIRAG:
    def __init__(self, model: str = "text-embedding-3-small"):
        self.embed_model = model
        self.documents = []
        self.embeddings = []
 
    def _embed(self, texts: list[str]) -> list[np.ndarray]:
        response = client.embeddings.create(model=self.embed_model, input=texts)
        return [np.array(item.embedding) for item in sorted(response.data, key=lambda x: x.index)]
 
    def index(self, docs: list[dict]):
        """docs: list of {"id": str, "text": str, "metadata": dict}"""
        self.documents = docs
        texts = [d["text"] for d in docs]
        raw_embs = self._embed(texts)
        # Normalize for fast dot-product similarity
        self.embeddings = [e / np.linalg.norm(e) for e in raw_embs]
 
    def retrieve(self, query: str, top_k: int = 4) -> list[dict]:
        q_emb = self._embed([query])[0]
        q_emb = q_emb / np.linalg.norm(q_emb)
        scores = [float(np.dot(q_emb, d)) for d in self.embeddings]
        top_idx = np.argsort(scores)[::-1][:top_k]
        return [{"doc": self.documents[i], "score": scores[i]} for i in top_idx]
 
    def answer(self, query: str, top_k: int = 4) -> dict:
        hits = self.retrieve(query, top_k)
        context = "\n\n".join(
            f"[{i+1}] {h['doc']['text']}" for i, h in enumerate(hits)
        )
 
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": "Answer using only the provided context. Cite sources with [1], [2], etc."},
                {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"}
            ]
        )
 
        return {
            "answer": response.choices[0].message.content,
            "sources": [h["doc"]["id"] for h in hits],
            "scores": [h["score"] for h in hits]
        }
 
# Usage
rag = OpenAIRAG()
rag.index([
    {"id": "policy-1", "text": "Returns accepted within 30 days.", "metadata": {"source": "handbook"}},
    {"id": "policy-2", "text": "Enterprise SLA guarantees 99.9% uptime.", "metadata": {"source": "contracts"}},
])
 
result = rag.answer("What is the refund window?")
print(result["answer"])
print(f"Sources: {result['sources']}")

Cost Calculation

import tiktoken
 
enc = tiktoken.get_encoding("cl100k_base")
 
def estimate_embedding_cost(texts: list[str], model: str = "text-embedding-3-small") -> dict:
    prices = {
        "text-embedding-3-small": 0.02 / 1_000_000,  # $0.02 per 1M tokens
        "text-embedding-3-large": 0.13 / 1_000_000,  # $0.13 per 1M tokens
        "text-embedding-ada-002": 0.10 / 1_000_000,
    }
 
    total_tokens = sum(len(enc.encode(t)) for t in texts)
    cost_usd = total_tokens * prices.get(model, 0.02 / 1_000_000)
 
    return {
        "total_tokens": total_tokens,
        "estimated_cost_usd": round(cost_usd, 6),
        "model": model,
    }
 
# 10,000 chunks of 500 tokens each
sample_texts = ["A " * 499 for _ in range(10000)]
print(estimate_embedding_cost(sample_texts))
# {'total_tokens': 5000000, 'estimated_cost_usd': 0.1, 'model': 'text-embedding-3-small'}

Common Mistakes / Pitfalls

Using ada-002 for new projects — text-embedding-3-small is better and cheaper
Not batching API calls — single-text calls at scale exhaust rate limits and cost 10x more per text than batched calls
Hardcoding dimensions=1536 — use native dimension reduction if you can tolerate minimal quality loss for storage savings
Not replacing newlines before embedding — newlines in the middle of text can degrade embedding quality
Comparing embeddings from different models — vectors are incomparable across models; re-embed if you switch

Best Practices

Use text-embedding-3-small as default; switch to text-embedding-3-large only if measured quality is insufficient
Always batch embed 100-2048 texts per API call — never call the API one text at a time in a loop
Normalize embeddings after generation and store normalized vectors — saves computation on every query
Use dimensions=256 to dimensions=512 for large-scale storage when cost or speed is a constraint
Cache embeddings in a vector store or Redis — never re-embed the same text twice

Key Takeaways

text-embedding-3-small is the recommended default: 1536 dimensions, MTEB score 62.3, $0.02/1M tokens
text-embedding-3-large provides higher quality (MTEB 64.6) at 3072 dimensions — use for research or high-precision tasks
Native dimensionality reduction via the dimensions parameter reduces storage by 80%+ with minimal quality loss
Batching up to 2048 texts per API call dramatically improves throughput and stays within rate limits
Always normalize embeddings to unit length before comparison — enables faster dot-product instead of cosine
Replacing newlines with spaces before embedding improves token representation quality
Cosine similarity between a query embedding and document embeddings is the core mechanism of semantic search
At 10M tokens per day, text-embedding-3-small costs ~$0.20/day — embedding at scale is extremely cheap compared to generation

Introduction#

Why This Matters#

API Setup#

Available Models#

Basic Embedding Call#

Batch Embedding (Critical for Cost Efficiency)#

Dimensionality Reduction#

Cosine Similarity and Search#

Full RAG Pipeline#

Cost Calculation#

Common Mistakes / Pitfalls#

Best Practices#

Key Takeaways#

Related reading