Published on

LLM Chaining Patterns — Sequential, Parallel, and Conditional Chains

Authors
  • Name
    Twitter

Introduction

Complex AI tasks rarely involve a single LLM call. Most production systems chain multiple LLM calls together: one to understand intent, another to gather information, a third to generate output, and a fourth to validate. This post covers production patterns for orchestrating these chains—from simple sequential chains to parallel processing, conditional routing, and map-reduce patterns for large documents.

Sequential Chain Pattern

Chain LLM outputs as inputs to subsequent calls. Simple, synchronous, and easy to debug.

import Anthropic from "@anthropic-ai/sdk";

interface ChainStep {
  id: string;
  prompt: (previousOutput?: string) => string;
  model?: string;
  maxTokens?: number;
}

interface ChainResult {
  stepId: string;
  output: string;
  tokensUsed: number;
  timeMs: number;
}

class SequentialChain {
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  async execute(steps: ChainStep[]): Promise<ChainResult[]> {
    const results: ChainResult[] = [];
    let previousOutput: string | undefined;

    for (const step of steps) {
      const startTime = Date.now();

      const prompt = step.prompt(previousOutput);

      const response = await this.client.messages.create({
        model: step.model || "claude-3-5-sonnet-20241022",
        max_tokens: step.maxTokens || 1024,
        messages: [{ role: "user", content: prompt }],
      });

      const output =
        response.content[0].type === "text" ? response.content[0].text : "";

      const result: ChainResult = {
        stepId: step.id,
        output,
        tokensUsed: response.usage.output_tokens,
        timeMs: Date.now() - startTime,
      };

      results.push(result);
      previousOutput = output;
    }

    return results;
  }
}

// Usage: Extract intent → gather context → generate response → validate
const intentExtractChain: ChainStep = {
  id: "extract-intent",
  prompt: (userInput) => `
Extract the intent from this user message in one sentence:
"${userInput}"
  `,
};

const contextGatherChain: ChainStep = {
  id: "gather-context",
  prompt: (intent) => `
Based on this intent: "${intent}"
Provide 3 relevant contextual facts:
  `,
};

const responseGenerationChain: ChainStep = {
  id: "generate-response",
  prompt: (context) => `
Using this context:
${context}

Generate a helpful response to the original query:
  `,
};

const validationChain: ChainStep = {
  id: "validate",
  prompt: (response) => `
Review this response for accuracy and helpfulness.
Rate it 1-5 and explain:
${response}
  `,
};

async function runSequentialExample() {
  const chain = new SequentialChain();
  const results = await chain.execute([
    intentExtractChain,
    contextGatherChain,
    responseGenerationChain,
    validationChain,
  ]);

  console.log(
    "Final validation:",
    results[results.length - 1].output
  );
}

Parallel Chain Pattern

Execute multiple independent chains concurrently and aggregate results.

interface ParallelChainConfig {
  chains: Array<{
    id: string;
    prompt: string;
    weight?: number; // For weighted averaging
  }>;
  aggregationStrategy: "average" | "concat" | "voting" | "custom";
}

class ParallelChain {
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  async executeParallel(config: ParallelChainConfig): Promise<string> {
    // Execute all chains concurrently
    const promises = config.chains.map((chainConfig) =>
      this.client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 512,
        messages: [{ role: "user", content: chainConfig.prompt }],
      })
    );

    const responses = await Promise.all(promises);

    // Extract text from responses
    const outputs = responses.map((response) =>
      response.content[0].type === "text" ? response.content[0].text : ""
    );

    // Aggregate based on strategy
    return this.aggregate(outputs, config);
  }

  private aggregate(
    outputs: string[],
    config: ParallelChainConfig
  ): string {
    switch (config.aggregationStrategy) {
      case "concat":
        return outputs.join("\n\n---\n\n");

      case "voting": {
        // For categorical outputs, pick most common
        const counts = new Map<string, number>();
        outputs.forEach((output) => {
          counts.set(output, (counts.get(output) || 0) + 1);
        });
        return Array.from(counts.entries()).sort(
          ([, a], [, b]) => b - a
        )[0][0];
      }

      case "average":
        // Numerical averaging
        const numbers = outputs
          .map((o) => parseFloat(o))
          .filter((n) => !isNaN(n));
        const avg = numbers.reduce((a, b) => a + b, 0) / numbers.length;
        return avg.toFixed(2);

      case "custom":
        // Implement domain-specific aggregation
        return this.customAggregate(outputs);

      default:
        return outputs[0];
    }
  }

  private customAggregate(outputs: string[]): string {
    // Application-specific logic
    return outputs[0];
  }
}

// Usage: Generate response from multiple perspectives
async function generateMultiPerspective(query: string): Promise<string> {
  const chain = new ParallelChain();

  const result = await chain.executeParallel({
    chains: [
      {
        id: "technical",
        prompt: `Answer from a technical expert''s perspective: ${query}`,
        weight: 1.0,
      },
      {
        id: "business",
        prompt: `Answer from a business strategy perspective: ${query}`,
        weight: 1.0,
      },
      {
        id: "user",
        prompt: `Answer from an end-user''s perspective: ${query}`,
        weight: 1.0,
      },
    ],
    aggregationStrategy: "concat",
  });

  return result;
}

Conditional Chain Pattern

Route to different prompts based on classification or decision logic.

type RouteKey = string;

interface ConditionalChainConfig {
  classificationPrompt: string;
  routes: Record<RouteKey, ChainStep[]>;
}

class ConditionalChain {
  private client: Anthropic;
  private sequentialChain: SequentialChain;

  constructor() {
    this.client = new Anthropic();
    this.sequentialChain = new SequentialChain();
  }

  async execute(
    userInput: string,
    config: ConditionalChainConfig
  ): Promise<string> {
    // Step 1: Classify input to determine route
    const classificationResponse = await this.client.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 100,
      messages: [
        {
          role: "user",
          content: `${config.classificationPrompt}\n\nInput: ${userInput}\n\nRespond with ONLY the route key.`,
        },
      ],
    });

    const route = (
      classificationResponse.content[0].type === "text"
        ? classificationResponse.content[0].text.trim()
        : ""
    ).toLowerCase();

    // Step 2: Select route and prepare steps with user input
    const selectedSteps = config.routes[route] || config.routes["default"] || [];

    const stepsWithInput = selectedSteps.map((step, idx) => ({
      ...step,
      prompt: (prev?: string) =>
        step.prompt(idx === 0 ? userInput : prev),
    }));

    // Step 3: Execute the selected chain
    const results = await this.sequentialChain.execute(stepsWithInput);

    return results[results.length - 1]?.output || "";
  }
}

// Usage: Route based on query type
async function routeBasedOnType(userQuery: string): Promise<string> {
  const chain = new ConditionalChain();

  const result = await chain.execute(userQuery, {
    classificationPrompt: `Classify this query as one of: technical_support, billing, feature_request, bug_report, general`,

    routes: {
      technical_support: [
        {
          id: "diagnose",
          prompt: (input) => `Diagnose the technical issue: ${input}`,
        },
        {
          id: "suggest_fix",
          prompt: (diagnosis) => `Based on diagnosis: ${diagnosis}\n\nSuggest a fix:`,
        },
      ],

      billing: [
        {
          id: "understand_issue",
          prompt: (input) => `What''s the billing question: ${input}`,
        },
        {
          id: "provide_answer",
          prompt: (issue) => `Answer the billing question: ${issue}`,
        },
      ],

      feature_request: [
        {
          id: "analyze",
          prompt: (input) => `Analyze this feature request: ${input}`,
        },
        {
          id: "suggest_implementation",
          prompt: (analysis) => `Implementation approach: ${analysis}`,
        },
      ],

      default: [
        {
          id: "respond",
          prompt: (input) => `Respond helpfully to: ${input}`,
        },
      ],
    },
  });

  return result;
}

Map-Reduce Chain for Long Documents

Process long documents by breaking into chunks, processing in parallel, then reducing.

interface MapReduceConfig {
  chunkSize: number;
  overlapSize: number;
  mapPrompt: (chunk: string, index: number) => string;
  reducePrompt: (mappedResults: string[]) => string;
}

class MapReduceChain {
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  private chunkDocument(
    document: string,
    chunkSize: number,
    overlapSize: number
  ): string[] {
    const chunks: string[] = [];
    let offset = 0;

    while (offset < document.length) {
      const chunk = document.substring(offset, offset + chunkSize);
      chunks.push(chunk);
      offset += chunkSize - overlapSize;
    }

    return chunks;
  }

  async execute(document: string, config: MapReduceConfig): Promise<string> {
    // Step 1: Map - process each chunk
    const chunks = this.chunkDocument(
      document,
      config.chunkSize,
      config.overlapSize
    );

    const mapPromises = chunks.map((chunk, idx) =>
      this.client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 512,
        messages: [{ role: "user", content: config.mapPrompt(chunk, idx) }],
      })
    );

    const mapResults = await Promise.all(mapPromises);
    const mappedOutputs = mapResults.map((response) =>
      response.content[0].type === "text" ? response.content[0].text : ""
    );

    // Step 2: Reduce - aggregate results
    const reducePrompt = config.reducePrompt(mappedOutputs);

    const finalResponse = await this.client.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 1024,
      messages: [{ role: "user", content: reducePrompt }],
    });

    return finalResponse.content[0].type === "text"
      ? finalResponse.content[0].text
      : "";
  }
}

// Usage: Summarize long document
async function summarizeLongDocument(document: string): Promise<string> {
  const chain = new MapReduceChain();

  const summary = await chain.execute(document, {
    chunkSize: 2000, // 2000 character chunks
    overlapSize: 500, // 500 character overlap between chunks
    mapPrompt: (chunk, idx) =>
      `Summarize this section (${idx + 1}): ${chunk}\n\nBe concise.`,
    reducePrompt: (summaries) =>
      `Combine these summaries into one coherent summary:\n${summaries.join(
        "\n---\n"
      )}`,
  });

  return summary;
}

Refine Chain Pattern

Iteratively improve answers through multiple refinement steps.

interface RefineConfig {
  initialPrompt: string;
  refinementPrompts: Array<{
    criterion: string;
    prompt: (currentAnswer: string) => string;
  }>;
  maxIterations: number;
}

class RefineChain {
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  async execute(
    userInput: string,
    config: RefineConfig
  ): Promise<{
    answer: string;
    iterations: number;
    refinements: string[];
  }> {
    let answer: string = "";
    const refinements: string[] = [];

    // Initial answer
    const initialResponse = await this.client.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 1024,
      messages: [
        {
          role: "user",
          content: `${config.initialPrompt}\n\nUser input: ${userInput}`,
        },
      ],
    });

    answer =
      initialResponse.content[0].type === "text"
        ? initialResponse.content[0].text
        : "";

    // Refine iteratively
    for (
      let i = 0;
      i < config.refinementPrompts.length &&
      i < config.maxIterations;
      i++
    ) {
      const refinement = config.refinementPrompts[i];

      const refinedResponse = await this.client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 1024,
        messages: [
          {
            role: "user",
            content: refinement.prompt(answer),
          },
        ],
      });

      const refined =
        refinedResponse.content[0].type === "text"
          ? refinedResponse.content[0].text
          : "";

      refinements.push(refined);
      answer = refined;
    }

    return {
      answer,
      iterations: refinements.length,
      refinements,
    };
  }
}

// Usage: Refine answer for clarity, accuracy, and completeness
async function refineAnswer(query: string): Promise<string> {
  const chain = new RefineChain();

  const result = await chain.execute(query, {
    initialPrompt: "Provide an initial answer to this question:",
    refinementPrompts: [
      {
        criterion: "clarity",
        prompt: (answer) =>
          `Make this clearer and more concise:\n${answer}`,
      },
      {
        criterion: "accuracy",
        prompt: (answer) =>
          `Check for accuracy and add citations if needed:\n${answer}`,
      },
      {
        criterion: "completeness",
        prompt: (answer) =>
          `Is this complete? Add any missing important points:\n${answer}`,
      },
    ],
    maxIterations: 3,
  });

  return result.answer;
}

Branch-and-Merge Pattern

Split processing into parallel branches, then merge results back together.

interface BranchConfig {
  id: string;
  prompt: (input: string) => string;
  weight?: number;
}

interface MergeStrategy {
  type: "weighted-concat" | "consensus" | "combine";
  weights?: Record<string, number>;
}

class BranchMergeChain {
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  async execute(
    input: string,
    branches: BranchConfig[],
    merge: MergeStrategy
  ): Promise<string> {
    // Execute all branches in parallel
    const promises = branches.map((branch) =>
      this.client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 512,
        messages: [{ role: "user", content: branch.prompt(input) }],
      })
    );

    const responses = await Promise.all(promises);

    // Extract outputs with branch IDs
    const outputs = responses.map((response, idx) => ({
      branchId: branches[idx].id,
      output:
        response.content[0].type === "text" ? response.content[0].text : "",
      weight: branches[idx].weight || 1.0,
    }));

    // Merge based on strategy
    return this.merge(outputs, merge);
  }

  private merge(
    outputs: Array<{ branchId: string; output: string; weight: number }>,
    strategy: MergeStrategy
  ): string {
    switch (strategy.type) {
      case "weighted-concat":
        return outputs
          .map((o) => `[${o.branchId}]\n${o.output}`)
          .join("\n\n---\n\n");

      case "consensus":
        // Find common themes across outputs
        return `Consensus across branches:\n${outputs[0].output}`;

      case "combine":
        // Merge insights from all branches
        const combined = outputs
          .map((o) => `- ${o.branchId}: ${o.output.split("\n")[0]}`)
          .join("\n");
        return combined;

      default:
        return outputs[0].output;
    }
  }
}

// Usage: Analyze product from multiple angles
async function analyzeProductMultiAngle(productDescription: string): Promise<string> {
  const chain = new BranchMergeChain();

  const result = await chain.execute(
    productDescription,
    [
      {
        id: "market-fit",
        prompt: (input) =>
          `Analyze market fit and competitive positioning: ${input}`,
        weight: 1.2,
      },
      {
        id: "user-value",
        prompt: (input) =>
          `What user value does this provide: ${input}`,
        weight: 1.0,
      },
      {
        id: "risks",
        prompt: (input) =>
          `What are the risks and challenges: ${input}`,
        weight: 0.8,
      },
    ],
    { type: "weighted-concat" }
  );

  return result;
}

Chain Debugging with Tracing

Instrument chains with detailed tracing for debugging and optimization.

interface TraceEvent {
  timestamp: Date;
  stepId: string;
  eventType: "start" | "complete" | "error";
  input?: string;
  output?: string;
  durationMs?: number;
  error?: string;
}

class ChainTracer {
  private trace: TraceEvent[] = [];
  private stepStartTimes = new Map<string, number>();

  startStep(stepId: string, input: string): void {
    this.trace.push({
      timestamp: new Date(),
      stepId,
      eventType: "start",
      input,
    });
    this.stepStartTimes.set(stepId, Date.now());
  }

  completeStep(stepId: string, output: string): void {
    const startTime = this.stepStartTimes.get(stepId) || Date.now();
    const durationMs = Date.now() - startTime;

    this.trace.push({
      timestamp: new Date(),
      stepId,
      eventType: "complete",
      output,
      durationMs,
    });

    this.stepStartTimes.delete(stepId);
  }

  recordError(stepId: string, error: Error): void {
    const startTime = this.stepStartTimes.get(stepId) || Date.now();
    const durationMs = Date.now() - startTime;

    this.trace.push({
      timestamp: new Date(),
      stepId,
      eventType: "error",
      error: error.message,
      durationMs,
    });

    this.stepStartTimes.delete(stepId);
  }

  getTrace(): TraceEvent[] {
    return this.trace;
  }

  getSummary(): {
    totalSteps: number;
    totalTime: number;
    stepTimes: Record<string, number>;
    errors: number;
  } {
    const stepTimes: Record<string, number> = {};
    let totalTime = 0;
    let errorCount = 0;

    this.trace.forEach((event) => {
      if (event.eventType === "complete" && event.durationMs) {
        stepTimes[event.stepId] =
          (stepTimes[event.stepId] || 0) + event.durationMs;
        totalTime += event.durationMs;
      }
      if (event.eventType === "error") {
        errorCount++;
      }
    });

    return {
      totalSteps: Object.keys(stepTimes).length,
      totalTime,
      stepTimes,
      errors: errorCount,
    };
  }
}

// Usage with tracing
async function tracedChainExecution() {
  const tracer = new ChainTracer();
  const client = new Anthropic();

  const steps: ChainStep[] = [
    { id: "step1", prompt: () => "First step" },
    { id: "step2", prompt: () => "Second step" },
  ];

  for (const step of steps) {
    tracer.startStep(step.id, "input");

    try {
      const response = await client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 512,
        messages: [{ role: "user", content: step.prompt() }],
      });

      const output =
        response.content[0].type === "text" ? response.content[0].text : "";
      tracer.completeStep(step.id, output);
    } catch (error) {
      tracer.recordError(step.id, error as Error);
    }
  }

  console.log("Chain Summary:", tracer.getSummary());
}

Chain Performance Optimization

Optimize chain performance by caching, parallelizing where possible, and reducing redundant calls.

interface CacheEntry {
  hash: string;
  result: string;
  timestamp: Date;
  ttlSeconds: number;
}

class OptimizedChain {
  private cache = new Map<string, CacheEntry>();
  private client: Anthropic;

  constructor() {
    this.client = new Anthropic();
  }

  private hashInput(input: string): string {
    // Simple hash for cache key
    let hash = 0;
    for (let i = 0; i < input.length; i++) {
      const char = input.charCodeAt(i);
      hash = (hash << 5) - hash + char;
      hash = hash & hash;
    }
    return hash.toString(36);
  }

  async callWithCache(prompt: string, ttlSeconds: number = 3600): Promise<string> {
    const hash = this.hashInput(prompt);
    const cached = this.cache.get(hash);

    // Check if cache is valid
    if (
      cached &&
      Date.now() - cached.timestamp.getTime() < cached.ttlSeconds * 1000
    ) {
      return cached.result;
    }

    // Cache miss or expired
    const response = await this.client.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 512,
      messages: [{ role: "user", content: prompt }],
    });

    const result =
      response.content[0].type === "text" ? response.content[0].text : "";

    // Store in cache
    this.cache.set(hash, {
      hash,
      result,
      timestamp: new Date(),
      ttlSeconds,
    });

    return result;
  }

  clearExpiredCache(): void {
    const now = Date.now();

    for (const [key, entry] of this.cache.entries()) {
      if (now - entry.timestamp.getTime() > entry.ttlSeconds * 1000) {
        this.cache.delete(key);
      }
    }
  }
}

Checklist

  • Use sequential chains for simple step-by-step workflows
  • Execute independent chains in parallel and aggregate results
  • Route to different chains based on input classification
  • Use map-reduce for processing long documents
  • Implement refinement loops to iteratively improve answers
  • Split complex tasks into parallel branches, then merge
  • Instrument chains with detailed tracing for debugging
  • Cache LLM responses to reduce redundant calls

Conclusion

Chaining patterns unlock the power of LLMs for complex, multi-step workflows. Sequential chains provide simplicity and debuggability. Parallel execution handles independent tasks efficiently. Conditional routing directs inputs to specialized chains. Map-reduce scales to large documents. Refinement loops improve quality iteratively. Branch-and-merge handles complex decomposition. Instrument everything with tracing to understand performance and debug failures. Cache aggressively to reduce redundant API calls. By mastering these patterns, you''ll build sophisticated AI systems that tackle real-world problems requiring multiple reasoning steps.