DSPy — Program LLMs Systematically Instead of Prompting (2026)

Introduction

Why DSPy Changes How You Build LLM Applications

DSPy (Declarative Self-improving Python) is a framework from Stanford that treats LLM programming the way machine learning treats model training: instead of hand-crafting prompts, you define the structure of your pipeline, collect examples, specify a metric, and let DSPy automatically optimize the prompts and few-shot examples for you.

The fundamental problem DSPy solves is that hand-crafted prompts are fragile. They break when you switch models, degrade as your data distribution shifts, and require expert knowledge to tune. DSPy replaces manual prompt engineering with automatic optimization — you write the logic, DSPy finds the best instructions.

Installation and Setup

pip install dspy-ai

import dspy
 
# Configure the LM backend
lm = dspy.OpenAI(model="gpt-4o", api_key="your-openai-key", max_tokens=1000)
dspy.settings.configure(lm=lm)
 
# Alternative: Use a local model via Ollama
# lm = dspy.OllamaLocal(model="llama3", max_tokens=1000)

Signatures: Declaring What You Want

A DSPy Signature declares the input and output fields of an LLM call — like a function signature for language models. DSPy uses these declarations to generate prompts automatically:

import dspy
 
# Simple signature using shorthand notation
class Sentiment(dspy.Signature):
    """Classify the sentiment of a customer review."""
    review: str = dspy.InputField(desc="The customer review text")
    sentiment: str = dspy.OutputField(desc="One of: positive, negative, neutral")
    confidence: float = dspy.OutputField(desc="Confidence score from 0.0 to 1.0")
 
# Use the signature directly with Predict
classifier = dspy.Predict(Sentiment)
result = classifier(review="The battery life is incredible, lasts all day!")
print(f"Sentiment: {result.sentiment}, Confidence: {result.confidence}")
 
# Multi-field extraction signature
class ExtractEntities(dspy.Signature):
    """Extract named entities from technical text."""
    text: str = dspy.InputField(desc="Input text to analyze")
    people: list[str] = dspy.OutputField(desc="List of person names mentioned")
    organizations: list[str] = dspy.OutputField(desc="List of organization names")
    technologies: list[str] = dspy.OutputField(desc="List of technologies or products")

Modules: Composable LLM Primitives

DSPy provides several built-in modules that implement common LLM patterns:

# dspy.Predict: Direct prediction with the signature
predict = dspy.Predict(Sentiment)
 
# dspy.ChainOfThought: Adds reasoning before the output
cot = dspy.ChainOfThought(Sentiment)
 
# dspy.ProgramOfThought: Generates and executes Python code to derive the answer
pot = dspy.ProgramOfThought(Sentiment)
 
# dspy.ReAct: Reasoning + Acting with tool use
react = dspy.ReAct(Sentiment, tools=[...])
 
# Comparing modules on the same signature
review = "The UI is clean but it crashed twice in one day."
 
direct_result = predict(review=review)
cot_result = cot(review=review)
 
print(f"Direct: {direct_result.sentiment}")
print(f"CoT reasoning: {cot_result.rationale}")
print(f"CoT sentiment: {cot_result.sentiment}")

Building a Multi-Step Pipeline

DSPy programs are Python classes that compose multiple modules:

class RAGPipeline(dspy.Module):
    """A retrieval-augmented generation pipeline for technical Q&A."""
 
    def __init__(self, retriever, num_passages: int = 3):
        super().__init__()
        self.retriever = retriever
        self.num_passages = num_passages
 
        # Define the generation step
        self.generate_answer = dspy.ChainOfThought(
            "context: list[str], question: str -> answer: str"
        )
 
        # Define an answer quality check step
        self.check_answer = dspy.Predict(
            "question: str, answer: str -> is_complete: bool, missing_info: str"
        )
 
    def forward(self, question: str) -> dspy.Prediction:
        # Step 1: Retrieve relevant passages
        passages = self.retriever(question, k=self.num_passages).passages
 
        # Step 2: Generate an answer from context
        prediction = self.generate_answer(
            context=passages,
            question=question
        )
 
        # Step 3: Check if the answer is complete
        quality = self.check_answer(
            question=question,
            answer=prediction.answer
        )
 
        return dspy.Prediction(
            answer=prediction.answer,
            rationale=prediction.rationale,
            is_complete=quality.is_complete,
            missing_info=quality.missing_info
        )
 
# Multi-hop reasoning pipeline
class MultiHopQA(dspy.Module):
    """Answer questions requiring multiple retrieval steps."""
 
    def __init__(self, retriever, hops: int = 2):
        super().__init__()
        self.retriever = retriever
        self.hops = hops
        self.generate_query = dspy.ChainOfThought("context: list[str], question: str -> query: str")
        self.generate_answer = dspy.ChainOfThought("context: list[str], question: str -> answer: str")
 
    def forward(self, question: str) -> dspy.Prediction:
        context = []
 
        for hop in range(self.hops):
            # Generate a refined search query based on accumulated context
            if hop == 0:
                query = question
            else:
                query_pred = self.generate_query(context=context, question=question)
                query = query_pred.query
 
            # Retrieve new passages
            new_passages = self.retriever(query, k=2).passages
            context.extend(new_passages)
 
        # Generate final answer from all retrieved context
        answer = self.generate_answer(context=context, question=question)
        return dspy.Prediction(answer=answer.answer, context=context)

Optimization with Teleprompters

The most powerful feature of DSPy is its ability to automatically optimize prompts and few-shot examples using labeled training data:

# Prepare training examples
train_examples = [
    dspy.Example(
        review="Absolutely love this product, works perfectly!",
        sentiment="positive",
        confidence=0.95
    ).with_inputs("review"),
 
    dspy.Example(
        review="Broken on arrival, terrible customer service.",
        sentiment="negative",
        confidence=0.98
    ).with_inputs("review"),
 
    dspy.Example(
        review="It's okay, does what it says.",
        sentiment="neutral",
        confidence=0.75
    ).with_inputs("review"),
    # Add 20-50 more examples for best results
]
 
# Define a metric function
def sentiment_metric(example, prediction, trace=None) -> float:
    """Return 1.0 for correct prediction, 0.0 otherwise."""
    label_correct = example.sentiment.lower() == prediction.sentiment.lower()
    # Bonus points if confidence is well-calibrated
    confidence_reasonable = 0.5 &lt;= float(prediction.confidence) &lt;= 1.0
    return float(label_correct) + (0.1 * float(confidence_reasonable))
 
# Optimize with BootstrapFewShot (fast, few-shot optimizer)
from dspy.teleprompt import BootstrapFewShot
 
optimizer = BootstrapFewShot(
    metric=sentiment_metric,
    max_bootstrapped_demos=4,   # Max few-shot examples to select
    max_labeled_demos=8,        # Max labeled examples from training set
    max_rounds=1
)
 
unoptimized_classifier = dspy.Predict(Sentiment)
optimized_classifier = optimizer.compile(
    unoptimized_classifier,
    trainset=train_examples
)
 
# The optimized classifier automatically uses the best few-shot examples
result = optimized_classifier(review="Fast shipping but the manual was confusing.")
print(f"Sentiment: {result.sentiment}, Confidence: {result.confidence}")

Advanced Optimizer: MIPRO

For more thorough optimization, MIPRO (Multi-prompt Instruction Proposal and Optimization) searches the instruction space:

from dspy.teleprompt import MIPROv2
 
# MIPROv2 optimizes both instructions AND few-shot examples
mipro_optimizer = MIPROv2(
    metric=sentiment_metric,
    auto="medium",              # "light", "medium", or "heavy" optimization budget
    num_threads=4,              # Parallel evaluation threads
    max_bootstrapped_demos=3,
    max_labeled_demos=5,
)
 
# This will take several minutes and make many LLM calls
optimized_pipeline = mipro_optimizer.compile(
    unoptimized_classifier,
    trainset=train_examples,
    num_trials=20,              # Number of candidate programs to evaluate
    requires_permission_to_run=False
)
 
# Save and load optimized programs
optimized_pipeline.save("optimized_sentiment.json")
 
# Load in production
production_classifier = dspy.Predict(Sentiment)
production_classifier.load("optimized_sentiment.json")

Evaluating DSPy Programs

from dspy.evaluate import Evaluate
 
# Create an evaluator
evaluator = Evaluate(
    devset=train_examples[:20],  # Use a held-out evaluation set
    metric=sentiment_metric,
    num_threads=4,
    display_progress=True
)
 
# Compare unoptimized vs optimized
baseline_score = evaluator(unoptimized_classifier)
optimized_score = evaluator(optimized_classifier)
 
print(f"Baseline accuracy: {baseline_score:.1%}")
print(f"Optimized accuracy: {optimized_score:.1%}")
print(f"Improvement: {optimized_score - baseline_score:.1%}")

Common Mistakes

Using DSPy without labeled data — DSPy optimization requires at least 20–50 labeled examples; without data it defaults to zero-shot behavior.
Poorly defined metrics — The metric function must reliably distinguish good from bad outputs; noisy metrics produce noisy optimization.
Not using .with_inputs() — Forgetting to mark input fields in training examples causes optimization errors.
Running MIPRO with a small budget — MIPRO needs many trials to find good instructions; auto="light" is often insufficient.
Ignoring module choice — dspy.Predict vs dspy.ChainOfThought makes a large accuracy difference on reasoning tasks.
Not saving optimized programs — Always save compiled programs to avoid re-running expensive optimization.

Best Practices

Collect at least 50 labeled training examples and 20 held-out evaluation examples before running optimization
Define your metric function carefully — it is the most important input to the optimizer
Start with BootstrapFewShot (fast, cheap) before graduating to MIPROv2 (thorough, expensive)
Use dspy.ChainOfThought for any task involving reasoning; use dspy.Predict for pure classification or extraction
Save compiled programs with .save() and load them in production to avoid re-optimization costs
Re-optimize when switching LLM providers or upgrading model versions — prompts optimized for GPT-4 may not work for GPT-4o

Key Takeaways

DSPy replaces hand-crafted prompts with automatically optimized instructions and few-shot examples derived from your data and metric
Signatures declare the structure of an LLM call (inputs, outputs, descriptions) without specifying the exact prompt text — DSPy generates the prompt
dspy.ChainOfThought significantly outperforms dspy.Predict on reasoning-heavy tasks by adding an intermediate rationale field
BootstrapFewShot is the fastest optimizer — it selects the best few-shot examples from your training set automatically
MIPROv2 searches both the instruction space and few-shot space, producing better results at higher computational cost
The metric function is the most critical design decision in DSPy — it must reliably distinguish correct from incorrect outputs
DSPy programs are saved as JSON and loaded in production, separating optimization (offline) from inference (online)
Re-optimization after switching LLM providers or model versions is recommended — optimal prompts are model-specific

Introduction#

Why DSPy Changes How You Build LLM Applications#

Installation and Setup#

Signatures: Declaring What You Want#

Modules: Composable LLM Primitives#

Building a Multi-Step Pipeline#

Optimization with Teleprompters#

Advanced Optimizer: MIPRO#

Evaluating DSPy Programs#

Common Mistakes#

Best Practices#

Key Takeaways#

Related reading