All 15 Background Jobs & Queues questions
More in Background Jobs & Queues
What is job idempotency, and why is it critical when designing background job processing? What is a cron job, and how would you schedule recurring background tasks in a Node.js application using node-cron or BullMQ's repeatable jobs? What is job concurrency in a worker process, and how do you configure it safely in BullMQ? How would you track and expose the progress of a long-running background job to the client that requested it? What is a 'stalled' job in BullMQ, and what causes it? What is a dead-letter queue (DLQ), and how should a Node.js job-processing system use one? How do you implement job prioritization in a Node.js queue system, and when is it necessary? What is the difference between a message queue (like RabbitMQ) and a job queue (like BullMQ), and when would you choose one over the other? How do you implement graceful shutdown for a Node.js background worker process to avoid losing or corrupting in-progress jobs? What is job de-duplication in a queue system, and how would you prevent the same logical job from being enqueued multiple times?
Advertisement
Node.js › Background Jobs & Queues

What are the trade-offs of using an in-memory job queue (like a simple array-based queue) versus a persistent, Redis-backed queue?

Intermediate

Answer

An in-memory queue is trivial to set up and has no external dependency, but every queued job is lost immediately if the process crashes or restarts, and it can't be shared across multiple worker processes or server instances -- fine for genuinely low-stakes, non-critical background work in a single-instance application. A Redis-backed (or database-backed) queue persists jobs durably across restarts, supports multiple independent worker processes pulling from the same shared queue, and provides built-in features like retries and delayed jobs, at the cost of an additional infrastructure dependency.
// Fragile in-memory queue -- jobs vanish on restart
const queue = [];
function addJob(job) { queue.push(job); }

// Durable alternative backed by Redis via BullMQ
const queue = new Queue('jobs', { connection: redisConnection });
await queue.add('task', jobData);
Real-world example A prototype internal tool initially uses a simple in-memory array as its job queue, but after a production incident where a server restart silently dropped a batch of pending customer notification jobs, the team migrates to a Redis-backed BullMQ queue to guarantee jobs survive restarts and deployments.

Common follow-ups: At what point in an application's growth does the added operational complexity of running Redis become clearly worth it?;What's a lightweight middle-ground option between a plain in-memory array and a full Redis-backed queue system?

Caching with Redis;Cloud & DevOps
All 15 Background Jobs & Queues questions