AI Output Validation — Schema Checking, Business Rules, and Safety Nets

Introduction

Your LLM returns a malformed JSON response that breaks your entire downstream pipeline. Or it returns a price that''s negative, violating your business rules. Or it contradicts the user''s input, confusing the AI app.

Output validation is your last line of defense before responses reach users or downstream systems. This post covers schema validation, business rule checking, and graceful fallbacks.

JSON Schema Validation with Zod
Retry with Error Feedback
Business Rule Validation
Semantic Validation: Output vs Input Consistency
Format Compliance Checking
Confidence Threshold Gating
Conclusion

JSON Schema Validation with Zod

Define strict schemas for LLM output and validate before using:

import { z } from "zod";

// Define expected output schema
const ProductRecommendationSchema = z.object({
  productId: z.string().uuid(),
  productName: z.string().min(1).max(200),
  price: z.number().positive(),
  discount: z.number().min(0).max(1),
  reasoning: z.string().min(10).max(1000),
  confidence: z.number().min(0).max(1),
});

type ProductRecommendation = z.infer<typeof ProductRecommendationSchema>;

interface ValidationResult<T> {
  valid: boolean;
  data?: T;
  errors: string[];
  strictMode: boolean;
}

class LLMOutputValidator {
  validateOutput<T>(
    output: unknown,
    schema: z.ZodSchema<T>,
    strictMode: boolean = false
  ): ValidationResult<T> {
    try {
      const parsed = schema.parse(output);
      return {
        valid: true,
        data: parsed,
        errors: [],
        strictMode,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        const errors = error.issues.map(
          (issue) =>
            `Field "${issue.path.join(".")}" ${issue.message}`
        );

        // In non-strict mode, attempt coercion
        if (!strictMode) {
          return this.attemptCoercion(output, schema, errors);
        }

        return {
          valid: false,
          errors,
          strictMode,
        };
      }

      return {
        valid: false,
        errors: ["Unknown validation error"],
        strictMode,
      };
    }
  }

  private attemptCoercion<T>(
    output: unknown,
    schema: z.ZodSchema<T>,
    originalErrors: string[]
  ): ValidationResult<T> {
    // Try to fix common issues (e.g., string numbers, missing fields)
    if (typeof output === "object" && output !== null) {
      const coerced = { ...output };

      // Coerce numeric strings
      for (const [key, value] of Object.entries(coerced)) {
        if (typeof value === "string" && !isNaN(Number(value))) {
          (coerced as Record<string, unknown>)[key] = Number(value);
        }
      }

      try {
        const parsed = schema.parse(coerced);
        return {
          valid: true,
          data: parsed,
          errors: originalErrors,
          strictMode: false,
        };
      } catch {
        return {
          valid: false,
          errors: originalErrors,
          strictMode: false,
        };
      }
    }

    return {
      valid: false,
      errors: originalErrors,
      strictMode: false,
    };
  }
}

export { LLMOutputValidator, ValidationResult, ProductRecommendationSchema };

Retry with Error Feedback

When validation fails, re-prompt the LLM with the specific error and ask it to fix:

interface RetryConfig {
  maxRetries: number;
  retryDelay: number; // milliseconds
  backoffMultiplier: number; // 1.5x delay each retry
}

class ValidationRetryHandler {
  private client: any; // Anthropic client

  constructor(apiKey: string, private config: RetryConfig) {
    this.client = new (require("@anthropic-ai/sdk")).Anthropic({ apiKey });
  }

  async callLLMWithValidation<T>(
    userPrompt: string,
    schema: z.ZodSchema<T>,
    systemPrompt?: string
  ): Promise<{ success: boolean; data?: T; finalError?: string }> {
    let currentPrompt = userPrompt;
    let lastError: string | undefined;

    for (let attempt = 0; attempt < this.config.maxRetries; attempt++) {
      // Call LLM
      const message = await this.client.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 1024,
        system:
          systemPrompt ||
          "You are a helpful assistant. Return responses as JSON.",
        messages: [{ role: "user", content: currentPrompt }],
      });

      const responseText =
        message.content[0].type === "text" ? message.content[0].text : "";

      // Try to parse JSON
      let parsed;
      try {
        parsed = JSON.parse(responseText);
      } catch {
        lastError = `Failed to parse JSON: ${responseText.slice(0, 100)}...`;

        // Retry with clarification
        currentPrompt = `${userPrompt}

IMPORTANT: You must respond with ONLY valid JSON, no markdown code blocks.`;

        await this.delay(this.config.retryDelay * Math.pow(this.config.backoffMultiplier, attempt));
        continue;
      }

      // Validate against schema
      const validator = new LLMOutputValidator();
      const result = validator.validateOutput(parsed, schema, false);

      if (result.valid) {
        return { success: true, data: result.data };
      }

      lastError = result.errors.join("; ");

      // Re-prompt with validation errors
      currentPrompt = `${userPrompt}

Previous response had errors:
${result.errors.map((e) => `- ${e}`).join("\n")}

Please fix these errors and return valid JSON.`;

      // Back off before retry
      if (attempt < this.config.maxRetries - 1) {
        await this.delay(
          this.config.retryDelay * Math.pow(this.config.backoffMultiplier, attempt)
        );
      }
    }

    return { success: false, finalError: lastError };
  }

  private delay(ms: number): Promise<void> {
    return new Promise((resolve) => setTimeout(resolve, ms));
  }
}

export { ValidationRetryHandler, RetryConfig };

Business Rule Validation

Check that outputs satisfy domain-specific business logic:

interface BusinessRuleViolation {
  rule: string;
  violated: boolean;
  reason: string;
}

interface BusinessRuleCheckResult {
  passed: boolean;
  violations: BusinessRuleViolation[];
}

class BusinessRuleValidator {
  async checkBusinessRules(
    output: ProductRecommendation
  ): Promise<BusinessRuleCheckResult> {
    const violations: BusinessRuleViolation[] = [];

    // Rule 1: Price must be positive
    if (output.price <= 0) {
      violations.push({
        rule: "price_must_be_positive",
        violated: true,
        reason: `Price ${output.price} is not positive`,
      });
    }

    // Rule 2: Discount must be between 0 and 1
    if (output.discount < 0 || output.discount > 1) {
      violations.push({
        rule: "discount_range",
        violated: true,
        reason: `Discount ${output.discount} is outside [0, 1]`,
      });
    }

    // Rule 3: Final price after discount must still be positive
    const finalPrice = output.price * (1 - output.discount);
    if (finalPrice <= 0) {
      violations.push({
        rule: "final_price_must_be_positive",
        violated: true,
        reason: `Final price ${finalPrice} is not positive`,
      });
    }

    // Rule 4: High confidence recommendations must have non-zero discount
    if (output.confidence > 0.9 && output.discount === 0) {
      violations.push({
        rule: "high_confidence_needs_incentive",
        violated: true,
        reason: "High confidence recommendations should offer a discount",
      });
    }

    // Rule 5: Reasoning length should match confidence
    const minReasoningLength = Math.round(output.confidence * 500);
    if (output.reasoning.length < minReasoningLength) {
      violations.push({
        rule: "reasoning_length_confidence_match",
        violated: true,
        reason: `Reasoning length ${output.reasoning.length} < expected ${minReasoningLength}`,
      });
    }

    return {
      passed: violations.length === 0,
      violations,
    };
  }
}

export { BusinessRuleValidator, BusinessRuleCheckResult, BusinessRuleViolation };

Semantic Validation: Output vs Input Consistency

Detect if the response contradicts or ignores the user''s input:

interface SemanticValidationResult {
  isConsistent: boolean;
  contradictions: string[];
  ignoredConstraints: string[];
}

class SemanticValidator {
  async validateConsistency(
    userInput: string,
    llmOutput: ProductRecommendation
  ): Promise<SemanticValidationResult> {
    const contradictions: string[] = [];
    const ignoredConstraints: string[] = [];

    // Extract constraints from user input
    const hasPriceLimit = /under\s+\$?(\d+)/i.test(userInput);
    const priceLimit = hasPriceLimit
      ? parseInt(userInput.match(/under\s+\$?(\d+)/i)?.[1] || "0")
      : null;

    if (priceLimit && llmOutput.price > priceLimit) {
      contradictions.push(
        `User requested products under $${priceLimit}, but recommended $${llmOutput.price}`
      );
    }

    // Check for exclusions
    const exclusionPattern = /(?:not|exclude|avoid)\s+(\w+)/gi;
    let match;
    while ((match = exclusionPattern.exec(userInput)) !== null) {
      const excluded = match[1].toLowerCase();
      if (llmOutput.productName.toLowerCase().includes(excluded)) {
        contradictions.push(
          `Product contains excluded term: ${excluded}`
        );
      }
    }

    // Check for minimum requirements
    const hasMinConfidenceReq = /at least|minimum|confidence.*(\d+)/i.test(userInput);
    if (hasMinConfidenceReq) {
      const minConfidence = parseInt(
        userInput.match(/(\d+)%?\s*(?:confidence|confident)/i)?.[1] || "0"
      ) / 100;
      if (llmOutput.confidence < minConfidence) {
        ignoredConstraints.push(
          `User requested minimum confidence ${minConfidence}, but output has ${llmOutput.confidence}`
        );
      }
    }

    return {
      isConsistent: contradictions.length === 0,
      contradictions,
      ignoredConstraints,
    };
  }
}

export { SemanticValidator, SemanticValidationResult };

Format Compliance Checking

Ensure output matches expected structure (length, field count, etc):

interface FormatCheckResult {
  compliant: boolean;
  violations: string[];
}

class FormatValidator {
  async checkFormatCompliance(
    output: ProductRecommendation,
    config: {
      minNameLength: number;
      maxNameLength: number;
      minReasoningLength: number;
      maxReasoningLength: number;
      requiredFields: string[];
    }
  ): Promise<FormatCheckResult> {
    const violations: string[] = [];

    // Check required fields
    for (const field of config.requiredFields) {
      if (!(field in output) || !output[field as keyof ProductRecommendation]) {
        violations.push(`Missing required field: ${field}`);
      }
    }

    // Check name length
    if (output.productName.length < config.minNameLength) {
      violations.push(
        `Product name too short: ${output.productName.length} < ${config.minNameLength}`
      );
    }
    if (output.productName.length > config.maxNameLength) {
      violations.push(
        `Product name too long: ${output.productName.length} > ${config.maxNameLength}`
      );
    }

    // Check reasoning length
    if (output.reasoning.length < config.minReasoningLength) {
      violations.push(
        `Reasoning too short: ${output.reasoning.length} < ${config.minReasoningLength}`
      );
    }
    if (output.reasoning.length > config.maxReasoningLength) {
      violations.push(
        `Reasoning too long: ${output.reasoning.length} > ${config.maxReasoningLength}`
      );
    }

    // Check no control characters
    const hasControlChars = /[\x00-\x1F]/.test(JSON.stringify(output));
    if (hasControlChars) {
      violations.push("Output contains control characters");
    }

    return {
      compliant: violations.length === 0,
      violations,
    };
  }
}

export { FormatValidator, FormatCheckResult };

Confidence Threshold Gating

Only show responses that meet a confidence threshold:

class ConfidenceGate {
  async shouldShowResponse(
    output: ProductRecommendation,
    threshold: number = 0.7
  ): Promise<{
    show: boolean;
    recommendation: "show" | "retry" | "fallback";
    confidence: number;
  }> {
    if (output.confidence >= threshold) {
      return {
        show: true,
        recommendation: "show",
        confidence: output.confidence,
      };
    }

    if (output.confidence >= threshold * 0.8) {
      // Close to threshold, worth retrying
      return {
        show: false,
        recommendation: "retry",
        confidence: output.confidence,
      };
    }

    return {
      show: false,
      recommendation: "fallback",
      confidence: output.confidence,
    };
  }

  buildFallbackResponse(userInput: string): ProductRecommendation {
    // Return sensible defaults when confidence is too low
    return {
      productId: "00000000-0000-0000-0000-000000000000",
      productName: "Unable to generate recommendation",
      price: 0,
      discount: 0,
      reasoning: "Our AI was not confident enough in a recommendation for your query. Please try rephrasing your request.",
      confidence: 0,
    };
  }
}

export { ConfidenceGate };

Conclusion

Output validation prevents bad data from corrupting downstream systems. Layer multiple checks: schema validation (Zod), business rule checking, semantic consistency, format compliance, and confidence gating. When validation fails, retry with error feedback, and fall back to sensible defaults when all retries exhaust.

Treat validation as a feature, not friction. Well-validated outputs mean faster iteration, fewer bugs, and more reliable AI products.