BullMQ in Production — Priority Queues, Rate Limiting, and Dead Letter Handling
Build reliable background job systems with BullMQ. Master priority queues, rate limiting, dead letter handling, and monitoring for production resilience.
webcoderspeed.com
14 articles
Build reliable background job systems with BullMQ. Master priority queues, rate limiting, dead letter handling, and monitoring for production resilience.
Users see stale prices. Admins update settings but the old value is served for 10 minutes. You delete a record but it keeps appearing. Cache invalidation is famously hard — and most implementations have subtle bugs that serve wrong data long after the source changed.
Cache stampede (a.k.a. thundering herd on TTL expiry) is one of the most dangerous failure modes in high-traffic systems. The moment your cache key expires, hundreds of simultaneous requests hammer your database — often killing it. Here''s how it happens, and exactly how to fix it.
Master Redis locks, Redlock algorithm, fencing tokens, and database advisory locks. Learn when distributed locks solve real problems.
Your service elects a leader to run background jobs. The network hiccups for 5 seconds. The old leader thinks it''s still leader. The new leader also thinks it''s leader. Both start processing the same queue. Now you have duplicate work, corrupted state, and a split-brain.
Implement exact-match and semantic caching with Redis to dramatically reduce LLM API calls, improving latency and cutting costs by 60% through intelligent cache invalidation.
Implement token-based rate limiting with per-user budgets, burst allowances, and cost anomaly detection to prevent runaway spending and ensure fair resource allocation.
Understand Redis Cluster architecture, consistent hashing, CROSSSLOT errors, hot slot detection, replication, and monitoring for production deployments.
Redis is full. Instead of failing gracefully, it starts silently evicting your most important cache keys — session tokens, rate limit counters, distributed locks. Your app behaves mysteriously until you realize Redis has been quietly deleting data. Here''s how to tame Redis eviction.
Redis evolved from a cache into a multi-model database: vector storage, time series, JSON, full-text search. Learn when to use Redis and modern patterns for 2026.
Master Redis patterns: cache-aside vs write-through caching, distributed locks with Lua, sliding window rate limiters, pub/sub, sorted sets for leaderboards, Redis Streams, and memory management.
Redis Streams offer persistence, consumer groups, and ordering without Kafka''s operational burden. Perfect for real-time activity feeds and notifications at scale.
Network partition splits your 3-node cluster into two halves. Both halves think they''re the primary. Both accept writes. Network heals. You have two diverged databases with conflicting data. This is split brain — one of the most dangerous failure modes in distributed systems.
Upstash brings Redis, Kafka, and QStash to serverless. Per-request pricing, no idle cost, perfect for Vercel, Netlify, and event-driven apps at scale.