The accounts service had MongoDB in it long before it had a queue in it. Then the work arrived that has no business happening inside the request that caused it — Stripe webhooks to reconcile, seat counts to recalculate, invoices to sync — and it needed somewhere to go.

The cheap answer was already installed. Agenda stores jobs in a MongoDB collection. We ran MongoDB. That is one connection string, no new service in the Terraform, and nothing new to be woken up about at three in the morning. I spent an afternoon wiring it up and then added Redis anyway.

This is the comparison I wish I had read first.

What BullMQ is

BullMQ is a job queue for Node that keeps its state in Redis. You write to it with a Queue, you consume from it with a Worker, and if you want to watch what is happening from a process that is doing neither, you attach QueueEvents.

The part that matters is underneath those classes. A queue is a Redis list of job IDs, a job is a hash, delayed jobs live in a sorted set keyed by the millisecond they become due, and every transition between those structures is a Lua script that Redis runs as a single command. Moving a job from waiting to active, incrementing its attempt counter and re-delaying it, promoting a delayed job when its time comes — none of those are a read, a decision in Node, and a write back. There is no window in which two workers have both decided the job is theirs.

The current release as I write this is 4.14.0, published yesterday. Version 4 landed in June, and the churn is worth knowing about before you go reading: if a tutorial tells you to construct a QueueScheduler alongside your worker, it is describing version 1. That class was removed in 2.0.0 and its two jobs — promoting delayed jobs and recovering stalled ones — moved into Worker. Half the BullMQ material on the internet still creates it.

A producer and a consumer, in full:

import { Queue, Worker, UnrecoverableError } from 'bullmq'
import IORedis from 'ioredis'

// Workers issue blocking commands, and ioredis will abort a command that is
// still outstanding after maxRetriesPerRequest. BullMQ requires null here and
// warns loudly if you forget.
const connection = new IORedis(process.env.REDIS_URL, {
  maxRetriesPerRequest: null,
})

export const invoices = new Queue('invoices', {
  connection,
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: 'exponential', delay: 2_000 },
    removeOnComplete: { age: 3_600, count: 1_000 },
    removeOnFail: { age: 7 * 24 * 3_600 },
  },
})

await invoices.add('sync', { subscriptionId }, { jobId: `sync:${subscriptionId}` })

const worker = new Worker<{ subscriptionId: string }>(
  'invoices',
  async (job) => {
    const result = await stripe.subscriptions.retrieve(job.data.subscriptionId)
    if (result.status === 'canceled') {
      // Nothing about this gets better on the fourth attempt.
      throw new UnrecoverableError('subscription is canceled')
    }
    return reconcile(result)
  },
  {
    connection,
    concurrency: 10,
    limiter: { max: 25, duration: 1_000 },
  },
)

process.on('SIGTERM', () => worker.close())

concurrency defaults to 1, which surprises people who assume a worker is a pool. limiter is a real rate limiter across every worker on that queue, not a per-process one, which is the reason it is in this example: Stripe's API has opinions about how fast you may talk to it, and I would rather express that once than write a token bucket.

What Agenda is

Agenda is a job scheduler for Node that keeps its state in a MongoDB collection — agendaJobs, unless you name it something else. You describe a job with define, and you put work on it with now, schedule or every.

import { Agenda } from 'agenda'

const agenda = new Agenda({
  db: { address: process.env.MONGO_URL, collection: 'agendaJobs' },
  processEvery: '5 seconds',
  maxConcurrency: 20,
})

agenda.define(
  'sync invoice',
  { concurrency: 10, lockLifetime: 60_000, priority: 'high' },
  async (job) => {
    const result = await stripe.subscriptions.retrieve(job.attrs.data.subscriptionId)
    await reconcile(result)
  },
)

await agenda.start()

await agenda.now('sync invoice', { subscriptionId })
await agenda.every('1 hour', 'sweep stale invoices', {}, { skipImmediate: true })

The API is genuinely nice to read. agenda.schedule('tomorrow at noon', ...) works, because Agenda parses English through human-interval and date.js, and I have never once had to look up what 'in 20 minutes' means.

The current release is 5.0.0, published in November 2022. That date is not a typo and I will come back to it.

The difference that actually decided it

Both libraries hand you a Job and both call your function. The mechanical difference is in how your function comes to be called at all.

A BullMQ worker sits inside BRPOPLPUSH on the waiting list. That is a blocking Redis command: the connection is parked in the server until something is pushed, and when a producer pushes a job ID the worker is handed it and wakes. Nothing is being asked repeatedly. The delay between add and the first line of the processor is a network round trip.

An Agenda instance runs a timer. Every processEvery — five seconds by default — it asks Mongo for work, and it asks per defined job name, with a findOneAndUpdate that matches documents that are due and unlocked (or whose lock has expired past lockLifetime) and stamps lockedAt in the same operation. That is a correct lock; Mongo's document-level atomicity is doing exactly the job Redis's single-threadedness does on the other side. But it happens on a clock, and the clock is the ceiling on how fast anything can start.

So the same eight jobs, enqueued at the same eight moments, come off the two queues very differently. On the blocking side each one leaves when it arrives. On the polling side they queue up against the next tick and leave in a clump, somewhere between zero and five seconds late, averaging two and a half.

For the nightly invoice sweep that is a rounding error and I would not have cared. For the job behind a customer clicking Upgrade and then staring at a seat count that has not moved, a p50 of two and a half seconds of pure waiting was the whole conversation. You can turn processEvery down, and then you are paying for a findOneAndUpdate per job name per instance per tick against a production Mongo cluster, forever, whether or not there is any work. That is a trade with no good end of it.

The retries I did not want to write

BullMQ gives you attempts and backoff as job options, with fixed and exponential built in, and UnrecoverableError to opt a specific failure out of the remaining attempts. A job that exhausts its attempts lands in the failed set with its stack trace, and stays there for as long as removeOnFail says.

Agenda has no equivalent. job.fail(reason) sets failReason, failedAt and increments failCount — and does not touch nextRunAt. A one-off job that throws is simply a document that failed once and will never run again. There is an unusually honest comment in Agenda's own source next to that code saying it "is not equipped to handle errors originating in user code".

So you write it. You listen for fail, read failCount, decide on a ceiling, compute a backoff, and reschedule the job yourself — and now the retry policy for your billing jobs is bespoke code in your repository that nobody has load-tested. It is not hard. It is just squarely in the category of thing I would rather import than own.

Recurring jobs muddy this slightly, and in a way that is worth being precise about: a recurring job's nextRunAt is computed when the run starts, so a failed occurrence does not stop the schedule. The job runs again on time. That occurrence is simply lost, and the only trace is failCount climbing.

What Agenda is genuinely better at

I would not want the above to read as a rout, because three of these mattered enough that I hesitated.

Jobs are documents you can query. During an incident, db.agendaJobs.find({ name: 'sync invoice', failCount: { $gt: 0 } }) in a Mongo shell is an answer in ten seconds, using a tool everyone on the team already has open. BullMQ's state is hashes and sorted sets with a key layout you are explicitly not meant to depend on, so you go through the library or you stand up Bull Board. That is a real operational difference and it is on Agenda's side.

Enqueue can be transactional. The job document and the business write it depends on can go into one Mongo transaction. With BullMQ, the write is in Mongo and the job is in Redis, so you have two stores and no transaction across them, and you inherit the usual problem: the row committed, the enqueue failed, nothing ever picks it up. The fix is the usual fix — make the job idempotent, or write an outbox — and both cost you something.

Recurring definitions do not drift. agenda.every() saves the job with type: 'single' and upserts on the name, so calling it at boot on all six replicas leaves you one document. BullMQ derives a repeatable job's key from name:jobId:endDate:tz:pattern, which means changing the schedule creates a new repeatable entry and quietly leaves the old one firing. You find out when something runs twice. The fix is to reconcile at boot — getRepeatableJobs(), compare against what the code declares, removeRepeatableByKey() for anything left over — and it is code I have now written twice at two companies, which suggests it should not be my code.

What it costs to make Redis hold jobs

The thing to internalise is that the moment BullMQ is in the service, Redis is a durable store and not a cache, and most Redis installations were configured by someone who believed the opposite.

BullMQ checks. It reads maxmemory-policy on connect and warns if it is not noeviction, because every other policy means Redis may decide to delete your job under memory pressure. It also refuses to start against anything older than Redis 5.0.0 outright. Beyond what it checks for you: persistence needs to be on and appropriate, and a managed Redis with a default eviction policy and no AOF is not a queue, however well the client library behaves.

The blast radius changes too. Mongo going down took the service with it anyway. Redis going down is a second thing that can take the service down, and it now needs the monitoring and the failover story to match.

The release cadence, which I have gone back and forth on

Agenda 5.0.0 was published in November 2022 and is still the current release twelve months later. In the same month I am writing this, BullMQ has shipped 4.13.0, 4.13.1, 4.13.2, 4.13.3, 4.14.0 and counting.

I want to be careful with this, because the obvious reading is lazy in both directions. A library that has stopped releasing may simply be finished, and I have been bitten more than once by a BullMQ minor release changing behaviour I was leaning on. Churn is not health and quiet is not death.

But @hokify/agenda exists — a TypeScript fork of Agenda, itself last published in December 2022 — and a fork is a signal about the upstream that no amount of charitable reading makes go away. For the thing that decides whether a customer gets billed, I want the queue's bug-fix cycle to be visibly alive.

Where I would still reach for Agenda

If the service is already on Mongo, there is no Redis anywhere near it, and the work is scheduled rather than reactive — nightly reports, hourly sweeps, the cleanup job nobody watches — then Agenda is the right answer and BullMQ is a second datastore you are adding for nothing. A five second floor on a job that runs at 3am is not a number anyone will ever measure. Add the transactional enqueue and the fact that your on-call can read the queue with the tools they already know, and it is not a consolation prize.

The moment any of that stops holding, the calculus flips hard. Reactive work where somebody is watching a spinner. Retries with backoff that you would otherwise hand-roll. A rate limit shared across every worker because a payment provider is on the other end. Jobs with dependencies, which BullMQ's FlowProducer expresses and Agenda has no concept of at all.

That was every requirement on the list, which is how a service that already had a perfectly good database ended up with a queue in a different one.

---

Versions as of writing: bullmq@4.14.0 (18 November 2023) and agenda@5.0.0 (7 November 2022). BullMQ 4.x targets Redis 5 and above; Agenda 5.x brings its own mongodb v4 driver.