5 articles
Your DLQ has 2 million messages. They''ve been there for 3 months. Nobody noticed. Those are failed orders, unpaid invoices, and unprocessed refunds — silently rotting. Here''s how to build a DLQ strategy that''s actually monitored, alerting, and self-healing.
At-least-once delivery is a guarantee, not a bug. Kafka, SQS, and RabbitMQ will deliver the same message more than once under failure conditions. Build idempotent consumers using Redis SET NX, database unique constraints, and Kafka transactional producers to handle duplicates safely.
Order created at 10:00. Order cancelled at 10:01. Your consumer processes them in reverse — cancellation arrives first, then creation "succeeds." The order is now in an invalid state. Event ordering bugs are subtle, expensive, and entirely avoidable.
Your queue has 50 million unprocessed messages. Consumers are processing 1,000/second. New messages arrive at 5,000/second. The backlog will never drain. Here''s how queue backlogs form, why they''re dangerous, and the patterns to prevent and recover from them.
Master message queue patterns: SQS FIFO for ordered processing, DLQ setup, Kafka for event streaming with consumer groups, BullMQ for in-process jobs, visibility timeout, deduplication, and backpressure.