Redis-Based Queues: BullMQ (Node.js) and Laravel Queues

Some work should not happen while a visitor waits: sending a welcome email, generating a PDF invoice, calling a slow third-party API. A job queue lets your app record "do this soon", reply to the visitor at once, and hand the work to a separate worker process. This guide covers two mature Redis-backed queues, BullMQ for Node.js and Laravel Queues for PHP, with the production patterns that matter: retries, scheduling, priorities, monitoring and safe deploys. It is written for your own server, such as a VPS.

1. Why a queue, and not cron?

Cron runs work on a timetable ("every hour"). A queue runs work in response to an event ("this user just registered"). Queues also give you what cron cannot:

Use cron for housekeeping on a fixed timetable, and a queue for anything triggered by users or other systems.

2. Set up Redis on your server

Both libraries use Redis as the broker: it is in memory, fast, and has the list and sorted-set structures queues need. On Ubuntu or Debian:

sudo apt update
sudo apt install redis-server
sudo systemctl enable --now redis-server
redis-cli ping    # replies PONG

Then edit /etc/redis/redis.conf and restart Redis with sudo systemctl restart redis-server:

# keep jobs across a crash or reboot
appendonly yes
appendfsync everysec

# a queue must never have its keys evicted
maxmemory-policy noeviction

# listen only on this machine
bind 127.0.0.1 -::1

Keep port 6379 closed to the internet. If an app on another server must connect, set a password (or an ACL user) and use a private network or an SSH tunnel. Valkey, a community fork of Redis, aims to be compatible; if you use it, test your queue library against it first.

3. BullMQ for Node.js

BullMQ is the successor to the older bull package. Install it with:

npm install bullmq

The producer: add jobs

// src/queues/email.js
import { Queue } from 'bullmq';

export const connection = {
  host: process.env.REDIS_HOST ?? '127.0.0.1',
  port: Number(process.env.REDIS_PORT ?? 6379),
};

export const emailQueue = new Queue('email', { connection });

export async function queueWelcomeEmail(userId) {
  await emailQueue.add('welcome-email', { userId }, {
    jobId: `welcome-${userId}`,            // a second add with the same id is ignored
    attempts: 3,
    backoff: { type: 'exponential', delay: 5000 },
    removeOnComplete: 1000,                // keep the last 1,000 finished jobs
    removeOnFail: 5000,
  });
}

Your registration route calls queueWelcomeEmail(user.id) and replies straight away. The jobId stops a double-click from sending two emails.

The worker: process jobs

// src/workers/email.worker.js
import { Worker } from 'bullmq';
import { connection } from '../queues/email.js';
import { sendWelcomeEmail } from '../lib/email.js';

const worker = new Worker('email', async (job) => {
  if (job.name === 'welcome-email') {
    await sendWelcomeEmail(job.data.userId);   // throw to trigger a retry
  }
}, {
  connection,
  concurrency: 5,                              // up to 5 jobs at once
  limiter: { max: 100, duration: 60_000 },     // at most 100 jobs a minute
});

worker.on('failed', (job, err) => console.error(`${job?.id} failed: ${err.message}`));

// finish the current jobs before exiting on deploy or shutdown
process.on('SIGTERM', async () => { await worker.close(); process.exit(0); });

Run the worker as its own process and keep it alive with PM2 (see PM2 process management):

pm2 start src/workers/email.worker.js --name email-worker

Delays, priorities, schedules and progress

// run in 24 hours
await emailQueue.add('reminder', { userId }, { delay: 24 * 60 * 60 * 1000 });

// a lower number runs first
await emailQueue.add('receipt', data, { priority: 1 });
await emailQueue.add('digest', data, { priority: 10 });

// every day at 09:00 (job schedulers, BullMQ 5.16 and later)
await emailQueue.upsertJobScheduler('daily-digest',
  { pattern: '0 9 * * *' },
  { name: 'daily-digest', data: {} });

// inside a long job, report progress for a progress bar
await job.updateProgress(50);

On older BullMQ versions you will see { repeat: { pattern } } on add(); job schedulers replace it.

Monitoring with Bull Board

npm install @bull-board/api @bull-board/express

import { createBullBoard } from '@bull-board/api';
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter.js';
import { ExpressAdapter } from '@bull-board/express';

const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');
createBullBoard({ queues: [new BullMQAdapter(emailQueue)], serverAdapter });

app.use('/admin/queues', requireAdminAuth, serverAdapter.getRouter());

The dashboard shows waiting, active, completed and failed jobs, with retry buttons. Always put it behind admin authentication: job data often contains personal details.

4. Laravel Queues

Laravel has a queue system built in. Point it at Redis and write job classes.

Configure

In .env:

QUEUE_CONNECTION=redis
REDIS_CLIENT=phpredis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

phpredis is the PHP Redis extension; on your own server install it with your distribution's package (for example php-redis on Ubuntu). If you can't install extensions, use the pure-PHP client instead: composer require predis/predis and REDIS_CLIENT=predis.

Create a job

php artisan make:job SendWelcomeEmail

<?php
namespace App\Jobs;

use App\Mail\WelcomeEmail;
use App\Models\User;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Mail;

class SendWelcomeEmail implements ShouldQueue
{
    use Queueable;

    public $tries = 3;
    public $timeout = 60;              // seconds before the worker kills the job
    public $backoff = [10, 30, 90];    // seconds between attempts

    public function __construct(public User $user) {}

    public function handle(): void
    {
        Mail::to($this->user->email)->send(new WelcomeEmail($this->user));
    }

    public function failed(\Throwable $e): void
    {
        Log::error("Welcome email failed for user {$this->user->id}: {$e->getMessage()}");
    }
}

This is the Laravel 11 and later job layout; older versions list the four traits Dispatchable, InteractsWithQueue, Queueable and SerializesModels instead.

Dispatch and run workers

SendWelcomeEmail::dispatch($user);                               // as soon as possible
SendWelcomeEmail::dispatch($user)->delay(now()->addMinutes(5));  // later
SendUrgentEmail::dispatch($user)->onQueue('high');               // a priority queue

php artisan queue:work redis --queue=high,default,low --tries=3 --timeout=60

The worker empties high before it looks at default and low. Set retry_after for the redis connection in config/queue.php to a few seconds more than your longest --timeout, or a slow job can be picked up twice. In production, run workers under Supervisor so they restart after a crash (see Supervisor for long-lived processes).

Chains and batches

use Illuminate\Support\Facades\Bus;

// one after another; if a step fails, the rest of the chain does not run
Bus::chain([
    new SendReceiptEmail($order),
    new UpdateInventory($order),
    new NotifyWarehouse($order),
])->catch(fn (\Throwable $e) => Log::error($e->getMessage()))->dispatch();

When steps don't depend on each other, dispatch them as separate jobs, or as a Bus::batch([...]) if you need to know when all of them have finished (batches need the job_batches table from php artisan make:queue-batches-table).

Horizon

composer require laravel/horizon
php artisan horizon:install
php artisan horizon

Horizon replaces queue:work with a supervised set of workers configured in config/horizon.php, and gives you a dashboard at /horizon with throughput, failed jobs and wait times. Run php artisan horizon under Supervisor, and restrict the dashboard in HorizonServiceProvider.

5. Running queues in production

6. Common mistakes

• Large payloads. Pass an ID or a storage key, not a file or a huge JSON blob; let the worker load the data.
• No retry policy. Transient failures become permanent. Use at least 2 or 3 attempts with backoff.
• Jobs that are not safe to repeat. A job can run twice after a crash or timeout. Make it idempotent: check "already sent?" before sending.
• Too many workers for the database. Each concurrent job may hold a database connection. Size your connection pool to match.
• No persistence or the wrong eviction policy. Without appendonly yes a crash loses queued jobs, and with an LRU policy Redis may delete them.

7. Running this on Domain India

A VPS is self-managed: you install and secure Redis, the workers and updates yourself. A small queue for one app runs comfortably on an entry-level plan.

On the App Platform, Node.js apps are detected automatically and anything else needs a Dockerfile; see getting started with the App Platform. To compare queue systems for other languages, see BullMQ vs Sidekiq vs Celery.

Frequently asked questions

Ready to run background workers? Compare VPS plans for a Redis-backed queue, or look at the App Platform if you would rather not manage a server. For help choosing, open a support ticket.