Few-Shot Learning with LLMs — Techniques and Python Examples (2026)

Introduction

Why Few-Shot Learning Is Powerful

Few-shot learning lets you adapt a general-purpose language model to a specialized task by providing a small number of labeled examples inside the prompt — no fine-tuning required. This approach is fast to iterate on, cheap to deploy, and often reaches 80–90% of fine-tuned model accuracy on many classification and extraction tasks.

The key insight from the GPT-3 paper is that large models are "meta-learners" — they can infer task structure from examples at inference time. Understanding how to select, format, and sequence those examples is the skill that separates good few-shot prompting from great few-shot prompting.

Zero-Shot vs One-Shot vs Few-Shot

Approach	Examples	When to Use
Zero-shot	0	Common tasks the model already knows well
One-shot	1	Format demonstration only
Few-shot	3–8	Custom labels, edge cases, domain-specific tasks
Many-shot	8+	Complex tasks with high variance, approaching fine-tuning

from openai import OpenAI
 
client = OpenAI()
 
# Zero-shot
zero_shot = "Classify the sentiment of this review: 'The delivery was two days late.' Label:"
 
# One-shot (format demonstration)
one_shot = """Classify sentiment as: positive, negative, or neutral.
 
Review: "I love this product!" -> positive
 
Review: "The delivery was two days late." -> """
 
# Few-shot (task learning)
few_shot = """Classify the sentiment of customer reviews.
 
Review: "I love this product!" -> positive
Review: "Terrible quality, broke in a week." -> negative
Review: "It arrived on time." -> neutral
Review: "Exceeded all my expectations!" -> positive
Review: "The delivery was two days late." -> """

Building a Few-Shot Classifier

def build_few_shot_classifier(
    examples: list[dict],
    label_key: str = "label",
    input_key: str = "text",
    task_description: str = "Classify the following text."
) -> callable:
    """
    Returns a callable classifier using few-shot prompting.
    examples: [{"text": "...", "label": "..."}, ...]
    """
    example_block = "\n".join([
        f'{input_key.title()}: "{ex[input_key]}" -> {ex[label_key]}'
        for ex in examples
    ])
 
    prompt_template = f"""{task_description}
 
{example_block}
 
{{input_key_title}}: "{{query}}" -> """
 
    def classify(query: str) -> str:
        prompt = prompt_template.format(
            input_key_title=input_key.title(),
            query=query
        )
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10
        )
        return response.choices[0].message.content.strip()
 
    return classify
 
# Build a ticket priority classifier
examples = [
    {"text": "Production database is down, all users affected", "label": "critical"},
    {"text": "User cannot reset their password", "label": "high"},
    {"text": "Dark mode not working on mobile", "label": "medium"},
    {"text": "Typo on the About page", "label": "low"},
    {"text": "Login page crashes for 30% of users", "label": "critical"},
]
 
classifier = build_few_shot_classifier(
    examples,
    task_description="Classify support ticket priority as: critical, high, medium, or low."
)
 
print(classifier("API rate limit errors affecting enterprise customers"))
# -> critical

Few-Shot Structured Data Extraction

Few-shot examples are especially effective for teaching custom extraction schemas:

import json
 
def build_extraction_prompt(examples: list[dict], text: str) -> str:
    """
    examples: [{"text": "...", "output": {...}}, ...]
    """
    lines = ["Extract structured information from text. Return valid JSON only.", ""]
 
    for ex in examples:
        lines.append(f'Text: "{ex["text"]}"')
        lines.append(f'JSON: {json.dumps(ex["output"])}')
        lines.append("")
 
    lines.append(f'Text: "{text}"')
    lines.append("JSON:")
    return "\n".join(lines)
 
extraction_examples = [
    {
        "text": "Alice Johnson, lead data engineer at Stripe, has been with the company 5 years.",
        "output": {"name": "Alice Johnson", "role": "lead data engineer",
                   "company": "Stripe", "tenure_years": 5}
    },
    {
        "text": "Dr. Raj Patel is a senior ML researcher at DeepMind, joining in 2021.",
        "output": {"name": "Dr. Raj Patel", "role": "senior ML researcher",
                   "company": "DeepMind", "tenure_years": 3}
    },
]
 
prompt = build_extraction_prompt(
    extraction_examples,
    "Sarah Kim, principal software architect at Cloudflare, celebrating her 8th work anniversary."
)
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    temperature=0,
    response_format={"type": "json_object"}
)
result = json.loads(response.choices[0].message.content)
print(result)

Selecting High-Quality Examples

The choice of examples has a larger impact on accuracy than the number of examples. Follow these principles:

import random
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
 
def get_embeddings(texts: list[str]) -> list[list[float]]:
    """Get embeddings for a list of texts."""
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts
    )
    return [item.embedding for item in response.data]
 
def select_similar_examples(
    query: str,
    example_pool: list[dict],
    n: int = 5,
    text_key: str = "text"
) -> list[dict]:
    """
    Select the n most semantically similar examples to the query.
    This is better than random selection for domain-specific tasks.
    """
    texts = [ex[text_key] for ex in example_pool]
    all_texts = [query] + texts
 
    embeddings = get_embeddings(all_texts)
    query_embedding = np.array(embeddings[0]).reshape(1, -1)
    example_embeddings = np.array(embeddings[1:])
 
    similarities = cosine_similarity(query_embedding, example_embeddings)[0]
    top_indices = np.argsort(similarities)[::-1][:n]
 
    return [example_pool[i] for i in top_indices]
 
# Usage: dynamically select most relevant examples for each query
query = "The app keeps freezing when uploading files larger than 1GB"
selected = select_similar_examples(query, examples, n=3)

Few-Shot for Code Generation

Teaching the model your team's coding style and conventions:

CODE_EXAMPLES = [
    {
        "task": "Validate email address",
        "code": '''def validate_email(email: str) -> bool:
    """Return True if email is valid, False otherwise."""
    import re
    pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$'
    return bool(re.match(pattern, email))'''
    },
    {
        "task": "Retry a function with exponential backoff",
        "code": '''def retry_with_backoff(func, max_retries: int = 3, base_delay: float = 1.0):
    """Retry func up to max_retries times with exponential backoff."""
    import time
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(base_delay * (2 ** attempt))'''
    },
]
 
def few_shot_code_gen(task: str) -> str:
    examples_block = "\n\n".join([
        f"Task: {ex['task']}\nCode:\n```python\n{ex['code']}\n```"
        for ex in CODE_EXAMPLES
    ])
 
    prompt = f"""Generate Python code following these examples. Include type hints and a docstring.
 
{examples_block}
 
Task: {task}
Code:"""
 
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1
    )
    return response.choices[0].message.content
 
print(few_shot_code_gen("Parse a CSV file and return rows as a list of dicts"))

Evaluating Few-Shot Performance

def evaluate_classifier(classifier: callable, test_cases: list[dict]) -> dict:
    """Measure few-shot classifier accuracy, precision per class."""
    from collections import defaultdict
 
    correct = 0
    per_class = defaultdict(lambda: {"tp": 0, "total": 0})
 
    for case in test_cases:
        predicted = classifier(case["text"]).lower().strip()
        expected = case["label"].lower().strip()
 
        per_class[expected]["total"] += 1
        if predicted == expected:
            correct += 1
            per_class[expected]["tp"] += 1
 
    accuracy = correct / len(test_cases)
    per_class_acc = {
        label: stats["tp"] / stats["total"]
        for label, stats in per_class.items()
    }
 
    return {"accuracy": accuracy, "per_class_accuracy": per_class_acc}

Common Mistakes

Random example selection — Examples should be diverse and representative, not randomly picked.
Label imbalance in examples — If 4 of 5 examples are "positive", the model will over-predict positive.
Examples that are too similar to each other — Cover the full space of variation in your data.
Inconsistent formatting — Examples must follow exactly the same format as the query.
Too many examples — Beyond 8–10 examples, context fills up and accuracy often plateaus or drops.
Poor-quality examples — One mislabeled example can significantly degrade performance.

Best Practices

Start with 3–5 examples and measure accuracy before adding more
Balance examples across all label classes
Select examples that cover edge cases and class boundaries
Use semantic similarity search to select the most relevant examples for each query dynamically
Keep formatting strictly consistent between examples and the query
Refresh examples periodically as your data distribution shifts

Key Takeaways

Few-shot learning adapts LLMs to custom tasks at inference time — no fine-tuning or GPU required
3–8 well-chosen examples typically yield 80–90% of fine-tuned model accuracy on classification tasks
Example quality matters more than quantity — one mislabeled example can degrade overall accuracy
Semantic similarity-based example selection outperforms random selection by 10–20% on specialized tasks
Label balance across examples is critical — imbalanced examples cause the model to over-predict majority classes
Few-shot prompting is most effective for classification, extraction, and format-constrained generation
Evaluate few-shot accuracy on a held-out test set before deploying to production
Dynamic few-shot (selecting examples per query via embeddings) scales better than a fixed example set

Introduction#

Why Few-Shot Learning Is Powerful#

Zero-Shot vs One-Shot vs Few-Shot#

Building a Few-Shot Classifier#

Few-Shot Structured Data Extraction#

Selecting High-Quality Examples#

Few-Shot for Code Generation#

Evaluating Few-Shot Performance#

Common Mistakes#

Best Practices#

Key Takeaways#

Related reading