7 questions found
What options does Redis offer for implementing a message queue, and how do they differ from each other?
Beginner
Redis offers simple lists combined with LPUSH and BRPOP for a basic first in first out queue, Pub/Sub for real time fire and forget messaging without persistence, and Streams for a more robust, log based messaging system with persistence, consumer groups, and message acknowledgment, each suited to different reliability and complexity needs.
-- Simple list based queue
LPUSH tasks 'process_order_123'
BRPOP tasks 30
Real-world example
A small internal tool uses a simple Redis list as a lightweight task queue, while a larger, more critical order processing system uses Redis Streams for the stronger delivery guarantees it needs.
Common follow-ups: Which of these three approaches offers the strongest delivery guarantees?;How do you decide which approach is right for a specific use case?
Lists in Redis;Streams
How do you build a simple work queue using a Redis list where multiple workers pull tasks to process?
Beginner
You have producers add new tasks to the queue using RPUSH, and have multiple worker processes each call BRPOP on the same queue key, with Redis guaranteeing that each individual task is only delivered to exactly one of the waiting workers, naturally distributing the workload across however many workers are currently running.
-- Producer
RPUSH tasks 'send_email_to_user_42'
-- Each worker
BRPOP tasks 0
Real-world example
An email sending service runs five worker processes all calling BRPOP on the same task queue, with Redis automatically distributing incoming email jobs across whichever workers happen to be available at that moment.
Common follow-ups: How do you scale the number of workers up or down based on queue length?;What happens if a worker crashes while holding a task it pulled from the queue?
Lists in Redis;Distributed Locks with Redis
What are the limitations of using a plain Redis list as a message queue, and when should you consider Redis Streams instead?
Intermediate
A plain list based queue does not retain any history of processed messages, offers no built-in way to acknowledge that a message was successfully processed, and provides no way to replay messages if something goes wrong, all of which Redis Streams specifically address, making Streams the better choice whenever you need message history, consumer groups, or reliable processing guarantees.
-- A list based queue loses the task entirely
-- once a worker pulls it, with no record kept
BRPOP tasks 0
Real-world example
A payment processing team switches from a simple list based queue to Redis Streams after realizing they needed the ability to replay and audit exactly which payment tasks were processed and when, something the simple list could not provide.
Common follow-ups: What specific features does Redis Streams add that a list cannot provide?;Is the added complexity of Streams always worth it for every use case?
Streams;Lists in Redis
How do you implement a priority queue pattern using Redis, where certain tasks should be processed before others?
Intermediate
You can use a sorted set instead of a plain list, storing each task with a score representing its priority, and have workers retrieve the highest or lowest priority task using ZPOPMIN or ZPOPMAX, ensuring that more urgent tasks are always processed ahead of lower priority ones regardless of the order they were originally added.
ZADD priority_queue 1 'urgent_task'
ZADD priority_queue 5 'normal_task'
ZPOPMIN priority_queue
Real-world example
A customer support ticketing system uses a sorted set based priority queue, ensuring tickets marked as urgent are always processed by available agents before lower priority tickets, regardless of when each ticket was originally submitted.
Common follow-ups: How do you handle two tasks that happen to have the exact same priority score?;Is ZPOPMIN an atomic operation safe for multiple concurrent workers?
Sets & Sorted Sets in Redis;Data Types
How would you implement a delayed task queue in Redis, where a task should only become available for processing after a specific future time?
Advanced
You use a sorted set where the score represents the timestamp at which each task should become due, with a background process periodically checking for and moving any tasks whose due timestamp has already passed into an actual processing queue, effectively implementing scheduled, delayed task execution using Redis's native data structures.
ZADD delayed_tasks 1725800000 'send_reminder_email'
-- A background process periodically checks:
ZRANGEBYSCORE delayed_tasks 0 <current_timestamp>
Real-world example
A reminder notification system schedules emails to be sent at a specific future time using a sorted set keyed by timestamp, with a background worker checking every few seconds for any reminders that have become due and moving them into the active processing queue.
Common follow-ups: How frequently should the background process check for due tasks?;How do you handle a task that needs to be cancelled before its scheduled time arrives?
Sets & Sorted Sets in Redis;Redis Performance Tuning & Benchmarking
What patterns help ensure exactly once or at least once processing semantics when using Redis as a message queue for critical tasks?
Advanced
For at least once processing, you use a reliable pattern like moving tasks into a dedicated processing list while they are being worked on, using RPOPLPUSH or LMOVE, so a crashed worker's tasks remain visible and can be retried, while achieving true exactly once processing generally requires additional application level logic like idempotency keys to safely handle the rare case of a task being processed more than once.
RPOPLPUSH tasks processing
-- Process the task
-- Use an idempotency key to safely handle potential duplicate processing
SET processed:task123 1 NX
Real-world example
A payment processing system combines a reliable queue pattern with idempotency keys, ensuring that even if a task is accidentally processed twice due to a worker crash and retry, the customer is never charged more than once.
Common follow-ups: What is an idempotency key, and how does it prevent duplicate processing side effects?;Why is true exactly once delivery considered very difficult to achieve in distributed systems?
Distributed Locks with Redis;Streams
How do you monitor the length and health of a Redis based queue to ensure workers are keeping up with incoming tasks?
Intermediate
You periodically check the queue's length using LLEN for a list based queue or similar commands for other structures, tracking this value over time to detect if it is steadily growing, which would indicate workers cannot keep up with the incoming task rate and additional workers or performance improvements may be needed.
LLEN tasks -- returns the current number of pending tasks
Real-world example
A monitoring dashboard tracks the length of a task queue over time, alerting the operations team when it detects the queue steadily growing, signaling that additional worker capacity is needed to keep up with demand.
Common follow-ups: What queue length would typically indicate a problem versus normal, healthy fluctuation?;How do you set up automated scaling of workers based on queue length?
Redis Monitoring & Observability;Redis Performance Tuning & Benchmarking