Most BullMQ tutorials stop at the happy path. This guide digs into worker crashes, stalled jobs, and Redis restarts — giving your Node.js team a production-hardened queue setup that actually holds up.

How to Build a Resilient Job Queue With BullMQ and Redis

A job queue that works on localhost is not the same thing as a job queue that works at 2 a.m. when a worker process silently dies mid-execution. The difference between those two realities is where most production incidents live.

BullMQ, built on Redis, is one of the most capable background job libraries in the Node.js ecosystem. But its default configuration is optimized for getting started quickly, not for surviving the chaos of a real production environment. This article walks through the failure modes that tutorials skip — and exactly how to handle them.

Why Redis and Why BullMQ

Redis is an in-memory data store with optional persistence. BullMQ uses Redis sorted sets, lists, and hashes to track job state across its lifecycle: waiting, active, completed, failed, and delayed. Because all state lives in Redis rather than in application memory, workers can be stateless and horizontally scalable.

BullMQ is the maintained successor to Bull. It brings first-class TypeScript support, worker concurrency controls, job flows (parent/child dependencies), and a cleaner event model. For any serious Node.js background-job workload, it is the right default choice today.

The Failure Modes Nobody Talks About

1. Worker Crashes Mid-Job

When a worker picks up a job, BullMQ moves it to the active state and starts a lock timer. If the worker process crashes before calling job.moveToCompleted() or job.moveToFailed(), the job remains locked in active. After the lock expires (lockDuration, default 30 seconds), BullMQ marks it as stalled and re-queues it.

This is correct behavior — but only if you have a stall checker running. Without one, stalled jobs pile up silently.

import { Worker, QueueScheduler } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis({ maxRetriesPerRequest: null });

// QueueScheduler is required for stalled job recovery and delayed jobs
const scheduler = new QueueScheduler('email-jobs', { connection });

const worker = new Worker(
  'email-jobs',
  async (job) => {
    await sendEmail(job.data);
  },
  {
    connection,
    concurrency: 5,
    lockDuration: 60000,       // extend if jobs take longer than 30s
    stalledInterval: 30000,    // how often to check for stalled jobs
    maxStalledCount: 2,        // move to failed after 2 stall cycles
  }
);

worker.on('failed', (job, err) => {
  console.error(`Job ${job?.id} failed:`, err.message);
});

The QueueScheduler is mandatory in BullMQ v1/v2 for handling delayed jobs and stall recovery. In BullMQ v3+, this responsibility moved into the Worker itself — know which version your project uses.

2. Redis Restarts and Data Loss

By default, Redis does not persist data to disk. A restart wipes every queue. For development, that is fine. For production, it is catastrophic.

Enable at minimum AOF (Append Only File) persistence in your redis.conf:

appendonly yes
appendfsync everysec

everysec is a good balance — you risk losing at most one second of writes on a hard crash, which is acceptable for most job queues. If you need zero data loss, use appendfsync always, but accept the write-throughput penalty.

For managed Redis (AWS ElastiCache, Redis Cloud, Upstash), enable persistence through the provider's dashboard and use a Multi-AZ or replicated tier. Never run a job queue against a Redis instance that has no replica and no persistence.

3. Jobs That Run Forever

A background job that never completes keeps its lock active and blocks a worker concurrency slot indefinitely. Always enforce a timeout inside your job handler:

const worker = new Worker('resize-jobs', async (job) => {
  await Promise.race([
    resizeImage(job.data.imageUrl),
    new Promise((_, reject) =>
      setTimeout(() => reject(new Error('Job timed out')), 25000)
    ),
  ]);
}, { connection });

Set your lockDuration slightly higher than your timeout so BullMQ does not declare the job stalled before your own timeout fires.

Retry Strategies That Make Sense

BullMQ supports exponential backoff out of the box. A naive retry-immediately strategy hammers a downstream service that is already struggling. Use backoff:

await queue.add('send-webhook', payload, {
  attempts: 5,
  backoff: {
    type: 'exponential',
    delay: 2000, // 2s, 4s, 8s, 16s, 32s
  },
  removeOnComplete: 500,   // keep last 500 completed jobs for inspection
  removeOnFail: 200,
});

removeOnComplete and removeOnFail are critical in high-throughput queues. Without them, Redis memory grows unbounded as completed job records accumulate.

Observability: You Cannot Fix What You Cannot See

A production queue needs metrics. At minimum, instrument:

Queue depth — jobs waiting in each queue
Processing rate — jobs completed per minute
Failure rate — failed jobs per minute
Lag — time between job creation and processing start

BullMQ exposes queue.getJobCounts() which returns counts for all states. Pipe this into your metrics system (Prometheus, Datadog, whatever you use) on a 10-15 second interval. Set alerts when the waiting count exceeds a threshold or when the failure rate spikes.

BullMQ Board is a solid open-source UI for visualizing queue state during debugging. It is not a substitute for metrics, but it accelerates incident diagnosis.

Connection Management Under Load

A common mistake is sharing a single Redis connection between the Queue (producer) and the Worker (consumer). BullMQ requires a dedicated connection per Worker instance because workers use blocking Redis commands. Always pass separate IORedis instances:

const producerConnection = new IORedis({ maxRetriesPerRequest: null });
const workerConnection = new IORedis({ maxRetriesPerRequest: null });

const queue = new Queue('jobs', { connection: producerConnection });
const worker = new Worker('jobs', handler, { connection: workerConnection });

Set maxRetriesPerRequest: null on all BullMQ connections. Without it, IORedis will throw on Redis commands that block longer than a few milliseconds, causing false errors during normal queue operation.

Graceful Shutdown

When deploying a new worker version, abruptly killing the process mid-job corrupts job state. Implement a graceful shutdown:

process.on('SIGTERM', async () => {
  await worker.close(); // waits for active jobs to finish, then exits
  await scheduler.close();
  process.exit(0);
});

worker.close() drains active jobs before shutting down. In Kubernetes, set your terminationGracePeriodSeconds long enough to accommodate your longest expected job duration.

Why This Matters for Your Project

Background jobs are load-bearing infrastructure. An email that never sends, a webhook that silently drops, or a file that never processes can cascade into support tickets, data inconsistencies, and lost revenue. The patterns above — stall recovery, Redis persistence, enforced timeouts, exponential backoff, clean shutdowns, and proper observability — are not optional refinements. They are the baseline for any queue you trust in production. Getting this right before your user base scales is significantly cheaper than retrofitting it after your first major incident.