Published on

Multi-Tenant AI Applications — Isolating Data, Prompts, and Models per Tenant

Authors
  • Name
    Twitter

Introduction

Multi-tenant AI applications must isolate data strictly while sharing infrastructure efficiently. A single compromised tenant or data leakage incident erodes user trust. This post covers production patterns for tenant isolation, per-tenant prompt customization, data isolation at every layer, cost tracking by tenant, and audit logging that satisfies compliance requirements.

Tenant-Specific System Prompts

Customize system prompts per tenant without code changes. Store tenant configurations centrally.

interface TenantConfig {
  tenantId: string;
  name: string;
  tier: "free" | "pro" | "enterprise";
  systemPromptTemplate: string;
  customInstructions?: string;
  allowedModels: string[];
  rateLimit: number; // requests per minute
  maxTokensPerRequest: number;
}

interface TenantPromptConfig extends TenantConfig {
  resolvedSystemPrompt: string;
}

class TenantPromptManager {
  private configs = new Map<string, TenantConfig>();

  registerTenant(config: TenantConfig): void {
    this.configs.set(config.tenantId, config);
  }

  getTenantPrompt(tenantId: string): TenantPromptConfig {
    const config = this.configs.get(tenantId);

    if (!config) {
      throw new Error(`Tenant not found: ${tenantId}`);
    }

    // Resolve system prompt template with tenant-specific values
    const resolvedSystemPrompt = this.resolvePrompt(
      config.systemPromptTemplate,
      {
        tenantName: config.name,
        tenantTier: config.tier,
        customInstructions: config.customInstructions || "",
      }
    );

    return {
      ...config,
      resolvedSystemPrompt,
    };
  }

  private resolvePrompt(
    template: string,
    context: Record<string, string>
  ): string {
    let resolved = template;

    for (const [key, value] of Object.entries(context)) {
      const placeholder = `{{${key}}}`;
      resolved = resolved.replace(new RegExp(placeholder, "g"), value);
    }

    return resolved;
  }
}

// Default system prompt template
const defaultPromptTemplate = `You are an AI assistant for {{tenantName}} ({{tenantTier}} tier customer).

Core instructions:
- Provide helpful, accurate responses
- Follow company guidelines: {{customInstructions}}
- Never reference other customers or their data
- Be transparent about your limitations

Remember: You are serving {{tenantName}} exclusively in this conversation.`;

// Usage
const tenantManager = new TenantPromptManager();

tenantManager.registerTenant({
  tenantId: "tenant-123",
  name: "Acme Corp",
  tier: "enterprise",
  systemPromptTemplate: defaultPromptTemplate,
  customInstructions: "Always use professional tone. Prioritize data security.",
  allowedModels: ["claude-3-5-sonnet-20241022"],
  rateLimit: 100,
  maxTokensPerRequest: 2048,
});

const tenantPrompt = tenantManager.getTenantPrompt("tenant-123");
console.log(tenantPrompt.resolvedSystemPrompt);

Per-Tenant Vector Store Namespaces

Isolate vector embeddings per tenant using namespaces to prevent semantic search leakage.

interface VectorDocument {
  id: string;
  tenantId: string;
  content: string;
  embedding: number[];
  metadata: Record<string, unknown>;
}

interface VectorSearchResult {
  documentId: string;
  similarity: number;
  content: string;
  metadata: Record<string, unknown>;
}

class TenantVectorStore {
  // Namespaced storage: tenantId -> documentId -> document
  private store = new Map<string, Map<string, VectorDocument>>();

  addDocument(doc: VectorDocument): void {
    if (!this.store.has(doc.tenantId)) {
      this.store.set(doc.tenantId, new Map());
    }

    const tenantStore = this.store.get(doc.tenantId)!;
    tenantStore.set(doc.id, doc);
  }

  searchWithinTenant(
    tenantId: string,
    queryEmbedding: number[],
    topK: number = 5
  ): VectorSearchResult[] {
    const tenantStore = this.store.get(tenantId);

    if (!tenantStore) {
      return [];
    }

    // Search only within tenant''s namespace
    const results = Array.from(tenantStore.values())
      .map((doc) => ({
        documentId: doc.id,
        similarity: this.cosineSimilarity(queryEmbedding, doc.embedding),
        content: doc.content,
        metadata: doc.metadata,
      }))
      .sort((a, b) => b.similarity - a.similarity)
      .slice(0, topK);

    return results;
  }

  deleteAllTenantDocuments(tenantId: string): void {
    // Complete data deletion for tenant
    this.store.delete(tenantId);
  }

  private cosineSimilarity(vec1: number[], vec2: number[]): number {
    let dotProduct = 0;
    let mag1 = 0;
    let mag2 = 0;

    for (let i = 0; i < vec1.length; i++) {
      dotProduct += vec1[i] * vec2[i];
      mag1 += vec1[i] * vec1[i];
      mag2 += vec2[i] * vec2[i];
    }

    return dotProduct / (Math.sqrt(mag1) * Math.sqrt(mag2));
  }
}

Tenant Data Isolation in RAG

Implement retrieval-augmented generation with strict tenant isolation at query time.

import Anthropic from "@anthropic-ai/sdk";

interface RAGQuery {
  tenantId: string;
  query: string;
  maxResults: number;
}

interface RAGContext {
  tenantId: string;
  retrievedDocuments: VectorSearchResult[];
  query: string;
}

class TenantIsolatedRAG {
  private vectorStore: TenantVectorStore;
  private client: Anthropic;

  constructor() {
    this.vectorStore = new TenantVectorStore();
    this.client = new Anthropic();
  }

  async answerWithRAG(ragQuery: RAGQuery): Promise<string> {
    // Step 1: Verify tenant has access
    const isValidTenant = await this.validateTenant(ragQuery.tenantId);
    if (!isValidTenant) {
      throw new Error(`Invalid tenant: ${ragQuery.tenantId}`);
    }

    // Step 2: Generate embedding for query (in production, use same model as indexing)
    const queryEmbedding = this.generateMockEmbedding(ragQuery.query);

    // Step 3: Search ONLY within tenant''s namespace
    const documents = this.vectorStore.searchWithinTenant(
      ragQuery.tenantId,
      queryEmbedding,
      ragQuery.maxResults
    );

    // Step 4: Build context from tenant''s documents only
    const context: RAGContext = {
      tenantId: ragQuery.tenantId,
      retrievedDocuments: documents,
      query: ragQuery.query,
    };

    // Step 5: Generate response with tenant-specific context
    return this.generateResponse(context);
  }

  private async generateResponse(context: RAGContext): Promise<string> {
    const relevantContext = context.retrievedDocuments
      .map((doc) => doc.content)
      .join("\n\n---\n\n");

    const systemPrompt = `You are answering for tenant ${context.tenantId}.
Use only the provided context. Do not reference other tenants or external knowledge not in the context.`;

    const userPrompt = `Context:\n${relevantContext}\n\nQuestion: ${context.query}`;

    const response = await this.client.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 1024,
      system: systemPrompt,
      messages: [{ role: "user", content: userPrompt }],
    });

    return response.content[0].type === "text" ? response.content[0].text : "";
  }

  private generateMockEmbedding(text: string): number[] {
    // In production, use real embedding service
    const dim = 1536;
    const embedding = new Array(dim);
    for (let i = 0; i < dim; i++) {
      embedding[i] = Math.sin(text.length + i) * 0.5 + 0.5;
    }
    return embedding;
  }

  private async validateTenant(tenantId: string): Promise<boolean> {
    // Check if tenant exists and is active
    return true; // Implement with database check
  }
}

Per-Tenant Rate Limits and Cost Tracking

Track API costs per tenant and enforce rate limits strictly.

interface TenantCostTracker {
  tenantId: string;
  monthlyBudget: number;
  currentSpend: number;
  tokenCount: number;
  lastReset: Date;
}

interface RequestCost {
  inputTokens: number;
  outputTokens: number;
  modelCost: number; // in cents
}

class TenantCostManager {
  private trackers = new Map<string, TenantCostTracker>();
  private costPerToken = {
    "claude-3-5-sonnet-20241022": {
      input: 0.003 / 1000, // cents per input token
      output: 0.015 / 1000, // cents per output token
    },
  };

  private rateLimiters = new Map<
    string,
    {
      requestCount: number;
      windowStart: Date;
      maxRequests: number;
    }
  >();

  initializeTenant(
    tenantId: string,
    monthlyBudgetCents: number,
    rateLimit: number
  ): void {
    this.trackers.set(tenantId, {
      tenantId,
      monthlyBudget: monthlyBudgetCents,
      currentSpend: 0,
      tokenCount: 0,
      lastReset: new Date(),
    });

    this.rateLimiters.set(tenantId, {
      requestCount: 0,
      windowStart: new Date(),
      maxRequests: rateLimit,
    });
  }

  checkRateLimit(tenantId: string): { allowed: boolean; remaining: number } {
    const limiter = this.rateLimiters.get(tenantId);

    if (!limiter) {
      return { allowed: false, remaining: 0 };
    }

    // Reset window if > 60 seconds
    const windowElapsed = Date.now() - limiter.windowStart.getTime();
    if (windowElapsed > 60000) {
      limiter.requestCount = 0;
      limiter.windowStart = new Date();
    }

    const allowed = limiter.requestCount < limiter.maxRequests;
    const remaining = limiter.maxRequests - limiter.requestCount;

    if (allowed) {
      limiter.requestCount++;
    }

    return { allowed, remaining };
  }

  recordCost(
    tenantId: string,
    model: string,
    inputTokens: number,
    outputTokens: number
  ): RequestCost {
    const tracker = this.trackers.get(tenantId);

    if (!tracker) {
      throw new Error(`Tenant not found: ${tenantId}`);
    }

    const costConfig = this.costPerToken[model as keyof typeof this.costPerToken];

    if (!costConfig) {
      throw new Error(`Unknown model: ${model}`);
    }

    const inputCost = inputTokens * costConfig.input;
    const outputCost = outputTokens * costConfig.output;
    const totalCost = inputCost + outputCost;

    tracker.currentSpend += totalCost;
    tracker.tokenCount += inputTokens + outputTokens;

    return {
      inputTokens,
      outputTokens,
      modelCost: totalCost,
    };
  }

  checkBudget(tenantId: string): {
    withinBudget: boolean;
    percentageUsed: number;
    remainingBudget: number;
  } {
    const tracker = this.trackers.get(tenantId);

    if (!tracker) {
      throw new Error(`Tenant not found: ${tenantId}`);
    }

    const percentageUsed = (tracker.currentSpend / tracker.monthlyBudget) * 100;
    const remainingBudget = Math.max(0, tracker.monthlyBudget - tracker.currentSpend);

    return {
      withinBudget: tracker.currentSpend <= tracker.monthlyBudget,
      percentageUsed,
      remainingBudget,
    };
  }

  resetMonthlyBudget(tenantId: string): void {
    const tracker = this.trackers.get(tenantId);

    if (tracker) {
      tracker.currentSpend = 0;
      tracker.tokenCount = 0;
      tracker.lastReset = new Date();
    }
  }
}

Per-Tenant Model Fine-Tuning

Allow enterprise tenants to fine-tune custom models while maintaining isolation.

interface TenantFineTuneConfig {
  tenantId: string;
  modelName: string;
  baseModel: string;
  trainingData: Array<{ prompt: string; completion: string }>;
  epochs: number;
  learningRate: number;
}

interface FineTuneStatus {
  tenantId: string;
  fineTuneId: string;
  status: "queued" | "training" | "completed" | "failed";
  trainingProgress: number; // 0-100
  modelName: string;
  completedAt?: Date;
  error?: string;
}

class TenantFineTuneManager {
  private fineTuneJobs = new Map<string, FineTuneStatus>();
  private tenantModels = new Map<string, string[]>(); // tenantId -> modelIds

  async submitFineTune(
    config: TenantFineTuneConfig
  ): Promise<FineTuneStatus> {
    // Validate tenant has enterprise tier
    const isEnterprise = await this.validateEnterpriseAccess(config.tenantId);

    if (!isEnterprise) {
      throw new Error(`Fine-tuning requires enterprise tier`);
    }

    const fineTuneId = `ft-${config.tenantId}-${Date.now()}`;

    const status: FineTuneStatus = {
      tenantId: config.tenantId,
      fineTuneId,
      status: "queued",
      trainingProgress: 0,
      modelName: config.modelName,
    };

    this.fineTuneJobs.set(fineTuneId, status);

    // Track model ownership by tenant
    if (!this.tenantModels.has(config.tenantId)) {
      this.tenantModels.set(config.tenantId, []);
    }
    this.tenantModels.get(config.tenantId)!.push(fineTuneId);

    // Queue fine-tuning job asynchronously
    this.queueFineTuningJob(config, fineTuneId);

    return status;
  }

  getFineTuneStatus(fineTuneId: string, tenantId: string): FineTuneStatus | null {
    const job = this.fineTuneJobs.get(fineTuneId);

    // Verify tenant owns this job
    if (job && job.tenantId !== tenantId) {
      throw new Error("Access denied: job belongs to different tenant");
    }

    return job || null;
  }

  listTenantModels(tenantId: string): string[] {
    return this.tenantModels.get(tenantId) || [];
  }

  private queueFineTuningJob(
    config: TenantFineTuneConfig,
    fineTuneId: string
  ): void {
    // Implementation: queue to training service
    console.log(`Queued fine-tuning job ${fineTuneId}`);
  }

  private async validateEnterpriseAccess(tenantId: string): Promise<boolean> {
    // Check tenant tier from database
    return true; // Implementation
  }
}

Cross-Tenant Data Leakage Prevention

Prevent data leakage through context injection, logging, and cache poisoning.

interface AuditLog {
  timestamp: Date;
  tenantId: string;
  action: string;
  resourceId: string;
  details: Record<string, unknown>;
}

class DataLeakagePrevention {
  private auditLogs: AuditLog[] = [];
  private responseCache = new Map<string, { response: string; tenantId: string }>();

  async callLLMSecurely(
    tenantId: string,
    systemPrompt: string,
    userMessage: string
  ): Promise<string> {
    // Step 1: Sanitize inputs for tenant leakage
    this.validatePromptForLeakage(systemPrompt, tenantId);
    this.validatePromptForLeakage(userMessage, tenantId);

    // Step 2: Create tenant-isolated context
    const secureSystemPrompt = `You are serving tenant: ${tenantId}
${systemPrompt}

CRITICAL: Never reference, mention, or infer information about other tenants or customers.`;

    // Step 3: Call LLM (implementation details)
    const response = await this.makeLLMCall(secureSystemPrompt, userMessage);

    // Step 4: Validate response for leaked information
    this.validateResponseForLeakage(response, tenantId);

    // Step 5: Log the interaction
    this.auditLogs.push({
      timestamp: new Date(),
      tenantId,
      action: "llm_call",
      resourceId: `call-${Date.now()}`,
      details: {
        messageLength: userMessage.length,
        responseLength: response.length,
      },
    });

    return response;
  }

  private validatePromptForLeakage(text: string, tenantId: string): void {
    // Check if prompt contains references to other tenant IDs
    const tenantIdPattern = /tenant-\d+/g;
    const matches = text.match(tenantIdPattern) || [];

    for (const match of matches) {
      if (match !== tenantId) {
        throw new Error(`Potential data leakage detected: reference to ${match}`);
      }
    }

    // Check for common leaked patterns
    const leakedPatterns = [
      /api[_-]?key/i,
      /secret/i,
      /password/i,
      /credit[_-]?card/i,
    ];

    for (const pattern of leakedPatterns) {
      if (pattern.test(text)) {
        console.warn(`Sensitive pattern detected in prompt: ${pattern}`);
      }
    }
  }

  private validateResponseForLeakage(response: string, tenantId: string): void {
    // Check response doesn''t contain other tenant references
    const otherTenantPattern = /tenant-\d+/g;
    const matches = response.match(otherTenantPattern) || [];

    for (const match of matches) {
      if (match !== tenantId) {
        throw new Error(
          `Response contains reference to other tenant: ${match}`
        );
      }
    }
  }

  private async makeLLMCall(
    systemPrompt: string,
    userMessage: string
  ): Promise<string> {
    // Implementation: call LLM
    return "Response";
  }

  getAuditLog(
    tenantId: string,
    startDate?: Date,
    endDate?: Date
  ): AuditLog[] {
    return this.auditLogs.filter((log) => {
      if (log.tenantId !== tenantId) return false;
      if (startDate && log.timestamp < startDate) return false;
      if (endDate && log.timestamp > endDate) return false;
      return true;
    });
  }
}

Tenant Onboarding Automation

Automate secure tenant provisioning with isolated resources.

interface TenantOnboardingRequest {
  tenantName: string;
  tier: "free" | "pro" | "enterprise";
  adminEmail: string;
}

interface OnboardedTenant {
  tenantId: string;
  name: string;
  tier: string;
  createdAt: Date;
  resourcesProvisioned: Record<string, string>; // resource name -> ID
}

class TenantOnboardingManager {
  async onboardTenant(
    request: TenantOnboardingRequest
  ): Promise<OnboardedTenant> {
    const tenantId = `tenant-${Date.now()}`;

    const resources: Record<string, string> = {};

    try {
      // 1. Create isolated database schema
      resources["database"] = await this.createDatabaseSchema(tenantId);

      // 2. Create vector store namespace
      resources["vectorstore"] = tenantId;

      // 3. Provision rate limiter
      resources["ratelimit"] = tenantId;

      // 4. Initialize cost tracker
      const budgetCents = this.calculateBudgetForTier(request.tier);
      resources["cost_tracker"] = `${tenantId}-cost`;

      // 5. Send welcome email
      await this.sendWelcomeEmail(request.adminEmail, tenantId);

      const tenant: OnboardedTenant = {
        tenantId,
        name: request.tenantName,
        tier: request.tier,
        createdAt: new Date(),
        resourcesProvisioned: resources,
      };

      return tenant;
    } catch (error) {
      // Cleanup if provisioning fails
      await this.cleanupFailedProvisioning(tenantId, resources);
      throw error;
    }
  }

  private async createDatabaseSchema(tenantId: string): Promise<string> {
    // Create isolated database with tenant-specific schema
    return `schema-${tenantId}`;
  }

  private calculateBudgetForTier(tier: string): number {
    const budgets = {
      free: 100, // 1 dollar
      pro: 10000, // 100 dollars
      enterprise: 1000000, // 10000 dollars
    };

    return budgets[tier as keyof typeof budgets] || 100;
  }

  private async sendWelcomeEmail(email: string, tenantId: string): Promise<void> {
    // Send onboarding email
    console.log(`Welcome email sent to ${email}`);
  }

  private async cleanupFailedProvisioning(
    tenantId: string,
    resources: Record<string, string>
  ): Promise<void> {
    console.log(`Cleaning up failed provisioning for ${tenantId}`);
  }
}

Audit Logging per Tenant

Log all tenant activities comprehensively for compliance and debugging.

interface DetailedAuditLog {
  logId: string;
  tenantId: string;
  timestamp: Date;
  eventType:
    | "api_call"
    | "data_access"
    | "configuration_change"
    | "deletion"
    | "access_denied";
  userId?: string;
  resourceType: string;
  resourceId: string;
  action: string;
  details: Record<string, unknown>;
  ipAddress?: string;
  userAgent?: string;
  status: "success" | "failure";
  error?: string;
}

class ComplianceAuditLogger {
  private logs: DetailedAuditLog[] = [];
  private logRetentionDays = 365; // 1 year retention

  logEvent(
    tenantId: string,
    eventType: DetailedAuditLog["eventType"],
    action: string,
    resourceType: string,
    resourceId: string,
    details?: Record<string, unknown>,
    userId?: string
  ): void {
    const log: DetailedAuditLog = {
      logId: `log-${Date.now()}`,
      tenantId,
      timestamp: new Date(),
      eventType,
      userId,
      resourceType,
      resourceId,
      action,
      details: details || {},
      status: "success",
    };

    this.logs.push(log);
  }

  exportTenantAuditLog(tenantId: string): string {
    // Export for compliance (GDPR, SOC 2, etc.)
    const tenantLogs = this.logs.filter((log) => log.tenantId === tenantId);

    const csv = [
      "Timestamp,Event Type,Action,Resource Type,Resource ID,User ID,Status,Error",
      ...tenantLogs.map((log) =>
        [
          log.timestamp.toISOString(),
          log.eventType,
          log.action,
          log.resourceType,
          log.resourceId,
          log.userId || "",
          log.status,
          log.error || "",
        ].join(",")
      ),
    ].join("\n");

    return csv;
  }

  cleanupExpiredLogs(): number {
    const cutoffDate = new Date(
      Date.now() - this.logRetentionDays * 24 * 60 * 60 * 1000
    );

    const beforeCount = this.logs.length;

    this.logs = this.logs.filter((log) => log.timestamp > cutoffDate);

    return beforeCount - this.logs.length;
  }
}

Checklist

  • Store tenant-specific system prompts with template variables
  • Isolate vector stores by tenant namespace to prevent semantic search leakage
  • Validate tenant access on every RAG query at query time
  • Enforce per-tenant rate limits and cost tracking
  • Require enterprise tier for custom model fine-tuning
  • Audit all cross-tenant data references in prompts and responses
  • Automate tenant provisioning with isolated resources
  • Log all tenant activities for compliance and debugging
  • Implement 1-year audit log retention for regulatory requirements

Conclusion

Multi-tenant AI systems require paranoia about data isolation. Isolate at every layer: prompts, vector stores, rate limiters, models, and cost tracking. Validate tenant ownership on every operation. Prevent leakage by sanitizing prompts for references to other tenants and validating responses. Automate onboarding to ensure consistent isolation from day one. Log everything comprehensively for compliance and incident investigation. By treating tenant isolation as a core architectural concern, you''ll build systems that scale safely and maintain the trust of your customers.