AI Multi-Tenancy Isolation — Securing Data, Prompts, and Models per Tenant

Introduction

Multi-tenant AI applications must isolate data strictly while sharing infrastructure efficiently. A single cross-tenant data leak destroys user trust and violates compliance requirements. This post covers production-grade patterns for tenant isolation across every layer: prompts, vector stores, RAG pipelines, rate limiters, and audit logs.

Why This Matters

When multiple customers share an AI backend, their data must never bleed across tenant boundaries. LLMs have a unique attack surface — prompt injection can trick a model into surfacing another tenant's context. Vector similarity search can accidentally retrieve documents from the wrong namespace. Shared caches can leak responses between tenants.

The consequences of getting this wrong are severe: GDPR violations, customer churn, and breach-of-contract claims. Getting isolation right from day one is far cheaper than retrofitting it after an incident.

Tenant-Specific System Prompts

Store per-tenant system prompt templates and resolve them at request time:

interface TenantConfig {
  tenantId: string;
  name: string;
  tier: 'free' | 'pro' | 'enterprise';
  systemPromptTemplate: string;
  customInstructions?: string;
  allowedModels: string[];
  rateLimit: number;
  maxTokensPerRequest: number;
}
 
class TenantPromptManager {
  private configs = new Map<string, TenantConfig>();
 
  registerTenant(config: TenantConfig): void {
    this.configs.set(config.tenantId, config);
  }
 
  getTenantSystemPrompt(tenantId: string): string {
    const config = this.configs.get(tenantId);
    if (!config) throw new Error(`Tenant not found: ${tenantId}`);
 
    return config.systemPromptTemplate
      .replace(/\{\{tenantName\}\}/g, config.name)
      .replace(/\{\{tenantTier\}\}/g, config.tier)
      .replace(/\{\{customInstructions\}\}/g, config.customInstructions ?? '');
  }
}
 
const DEFAULT_TEMPLATE = `You are an AI assistant for {{tenantName}} ({{tenantTier}} tier).
Instructions: {{customInstructions}}
Never reference other customers or their data.`;
 
const manager = new TenantPromptManager();
manager.registerTenant({
  tenantId: 'tenant-123',
  name: 'Acme Corp',
  tier: 'enterprise',
  systemPromptTemplate: DEFAULT_TEMPLATE,
  customInstructions: 'Use professional tone. Prioritize data security.',
  allowedModels: ['claude-3-5-sonnet-20241022'],
  rateLimit: 100,
  maxTokensPerRequest: 2048,
});

Per-Tenant Vector Store Namespaces

Isolate vector embeddings per tenant to prevent semantic search leakage:

interface VectorDocument {
  id: string;
  tenantId: string;
  content: string;
  embedding: number[];
  metadata: Record<string, unknown>;
}
 
class TenantVectorStore {
  // tenantId -> documentId -> document
  private store = new Map<string, Map<string, VectorDocument>>();
 
  addDocument(doc: VectorDocument): void {
    if (!this.store.has(doc.tenantId)) {
      this.store.set(doc.tenantId, new Map());
    }
    this.store.get(doc.tenantId)!.set(doc.id, doc);
  }
 
  search(tenantId: string, queryEmbedding: number[], topK = 5) {
    const tenantStore = this.store.get(tenantId);
    if (!tenantStore) return [];
 
    return Array.from(tenantStore.values())
      .map((doc) => ({
        id: doc.id,
        content: doc.content,
        score: this.cosineSimilarity(queryEmbedding, doc.embedding),
      }))
      .sort((a, b) => b.score - a.score)
      .slice(0, topK);
  }
 
  deleteAllForTenant(tenantId: string): void {
    this.store.delete(tenantId);
  }
 
  private cosineSimilarity(a: number[], b: number[]): number {
    let dot = 0, magA = 0, magB = 0;
    for (let i = 0; i < a.length; i++) {
      dot += a[i] * b[i];
      magA += a[i] ** 2;
      magB += b[i] ** 2;
    }
    return dot / (Math.sqrt(magA) * Math.sqrt(magB));
  }
}

Tenant-Isolated RAG Pipeline

Retrieval-augmented generation with strict namespace enforcement at query time:

import Anthropic from '@anthropic-ai/sdk';
 
class TenantIsolatedRAG {
  private vectorStore = new TenantVectorStore();
  private client = new Anthropic();
 
  async answer(tenantId: string, query: string): Promise<string> {
    await this.assertValidTenant(tenantId);
 
    const queryEmbedding = this.embed(query);
    const docs = this.vectorStore.search(tenantId, queryEmbedding, 5);
 
    const context = docs.map((d) => d.content).join('\n\n---\n\n');
 
    const response = await this.client.messages.create({
      model: 'claude-3-5-sonnet-20241022',
      max_tokens: 1024,
      system: `You answer questions for tenant ${tenantId} only. Use the provided context.`,
      messages: [{ role: 'user', content: `Context:\n${context}\n\nQuestion: ${query}` }],
    });
 
    return response.content[0].type === 'text' ? response.content[0].text : '';
  }
 
  private embed(text: string): number[] {
    // Replace with real embedding service (e.g. OpenAI, Cohere)
    return Array.from({ length: 1536 }, (_, i) => Math.sin(text.length + i) * 0.5);
  }
 
  private async assertValidTenant(tenantId: string): Promise<void> {
    // Query your database to confirm tenant is active
    if (!tenantId) throw new Error('Invalid tenant');
  }
}

Per-Tenant Rate Limiting and Cost Tracking

Enforce budgets and rate limits in a single middleware layer:

interface TenantUsage {
  tenantId: string;
  monthlyBudgetCents: number;
  spentCents: number;
  requestsThisMinute: number;
  windowStart: number;
  rateLimit: number;
}
 
class TenantCostManager {
  private usage = new Map<string, TenantUsage>();
 
  init(tenantId: string, monthlyBudgetCents: number, rateLimit: number) {
    this.usage.set(tenantId, {
      tenantId,
      monthlyBudgetCents,
      spentCents: 0,
      requestsThisMinute: 0,
      windowStart: Date.now(),
      rateLimit,
    });
  }
 
  checkAndConsume(tenantId: string): { allowed: boolean; reason?: string } {
    const u = this.usage.get(tenantId);
    if (!u) return { allowed: false, reason: 'Tenant not found' };
 
    // Reset rate-limit window
    if (Date.now() - u.windowStart > 60_000) {
      u.requestsThisMinute = 0;
      u.windowStart = Date.now();
    }
 
    if (u.requestsThisMinute >= u.rateLimit) {
      return { allowed: false, reason: 'Rate limit exceeded' };
    }
    if (u.spentCents >= u.monthlyBudgetCents) {
      return { allowed: false, reason: 'Monthly budget exceeded' };
    }
 
    u.requestsThisMinute++;
    return { allowed: true };
  }
 
  recordCost(tenantId: string, inputTokens: number, outputTokens: number) {
    const u = this.usage.get(tenantId);
    if (!u) return;
    // claude-3-5-sonnet pricing: $3/M input, $15/M output
    u.spentCents += (inputTokens * 0.0003 + outputTokens * 0.0015) / 100;
  }
}

Cross-Tenant Leakage Prevention

Validate prompts and responses to catch accidental or injected cross-tenant references:

class LeakagePrevention {
  private readonly SENSITIVE = [/api[_-]?key/i, /password/i, /credit.?card/i];
 
  validateInput(text: string, tenantId: string): void {
    // Reject references to other tenants
    const others = (text.match(/tenant-[a-z0-9-]+/gi) ?? [])
      .filter((t) => t.toLowerCase() !== tenantId.toLowerCase());
 
    if (others.length > 0) {
      throw new Error(`Prompt references foreign tenants: ${others.join(', ')}`);
    }
 
    for (const pattern of this.SENSITIVE) {
      if (pattern.test(text)) {
        console.warn(`Sensitive pattern found in input: ${pattern}`);
      }
    }
  }
 
  validateOutput(response: string, tenantId: string): void {
    const others = (response.match(/tenant-[a-z0-9-]+/gi) ?? [])
      .filter((t) => t.toLowerCase() !== tenantId.toLowerCase());
 
    if (others.length > 0) {
      throw new Error(`Response contains foreign tenant reference: ${others.join(', ')}`);
    }
  }
}

Compliance Audit Logging

Log every tenant interaction for GDPR and SOC 2 compliance:

interface AuditEntry {
  logId: string;
  tenantId: string;
  timestamp: string;
  eventType: 'api_call' | 'data_access' | 'config_change' | 'deletion' | 'access_denied';
  action: string;
  resourceId: string;
  status: 'success' | 'failure';
  error?: string;
}
 
class ComplianceLogger {
  private logs: AuditEntry[] = [];
 
  log(tenantId: string, eventType: AuditEntry['eventType'], action: string, resourceId: string, status: AuditEntry['status'] = 'success', error?: string) {
    this.logs.push({
      logId: `log-${Date.now()}-${Math.random().toString(36).slice(2)}`,
      tenantId,
      timestamp: new Date().toISOString(),
      eventType,
      action,
      resourceId,
      status,
      error,
    });
  }
 
  exportCSV(tenantId: string): string {
    const rows = this.logs
      .filter((l) => l.tenantId === tenantId)
      .map((l) => [l.timestamp, l.eventType, l.action, l.resourceId, l.status, l.error ?? ''].join(','));
 
    return ['timestamp,eventType,action,resourceId,status,error', ...rows].join('\n');
  }
 
  purgeExpired(retentionDays = 365): number {
    const cutoff = Date.now() - retentionDays * 86_400_000;
    const before = this.logs.length;
    this.logs = this.logs.filter((l) => new Date(l.timestamp).getTime() > cutoff);
    return before - this.logs.length;
  }
}

Common Mistakes

Shared cache without tenant keys: Caching LLM responses by content hash alone leaks data between tenants. Always namespace cache keys by tenantId.
No ownership check on fine-tuned models: A tenant querying another tenant's fine-tuned model by guessing an ID. Always verify model ownership before inference.
Logging full prompt text: Audit logs that include raw prompts can store cross-tenant data in a shared log store. Log metadata (lengths, token counts) not content.
Missing rate-limit window reset: Forgetting to reset the sliding window means rate limits never recover, starving legitimate traffic.
Hard-coded tenant IDs in system prompts: Never embed tenant identifiers as string literals in code. Resolve them dynamically at request time.

Best Practices

Namespace every persistent store (vector DB, cache, object storage) by tenantId as the primary key prefix.
Run tenant validation as the first step of every request handler, before any data access.
Use database row-level security (RLS) in Postgres to enforce tenant isolation at the query layer.
Emit cost metrics per tenant to a time-series store (Prometheus, CloudWatch) for billing and anomaly detection.
Store prompt templates in your database, not in code, so tenant customizations never require a deploy.
Test cross-tenant isolation with automated integration tests that attempt to access another tenant's resources.

Key Takeaways

Tenant isolation must be enforced at every layer: prompts, vector stores, caches, databases, and rate limiters — not just at the API gateway.
Per-tenant system prompt templates stored in a database allow customization without code changes or redeployments.
Vector store namespacing by tenant ID is the primary defence against cross-tenant semantic search leakage.
Rate limiting and monthly budget enforcement must both be active simultaneously; one without the other leaves a gap.
Leakage prevention requires validating both input prompts and output responses for foreign tenant references.
Compliance audit logs should record metadata (token counts, resource IDs, status) rather than raw prompt content to avoid storing leaked data in logs.
Automated tenant provisioning that sets up isolated resources from day one is cheaper than retrofitting isolation later.
A 365-day audit log retention policy satisfies most GDPR, SOC 2, and HIPAA audit trail requirements.

Introduction#

Why This Matters#

Tenant-Specific System Prompts#

Per-Tenant Vector Store Namespaces#

Tenant-Isolated RAG Pipeline#

Per-Tenant Rate Limiting and Cost Tracking#

Cross-Tenant Leakage Prevention#

Compliance Audit Logging#

Common Mistakes#

Best Practices#

Key Takeaways#

Related reading