Replay failed jobs

When a job fails its last attempt, Delay moves it to the dead letter queue (the Dead letters tab) instead of dropping it. Fix the cause, then replay the job to send the same request again.

When a job is dead-lettered

A job is dead-lettered when:

• its last retry fails (with the default delays, that’s the 4th attempt), or
• it fails once on a queue with Enable automatic retries turned off, or
• Delay’s worker fails while running it (rare).

At that point Delay also sends the job’s callback with status set to dead_lettered and, if the queue has alerts on, a failure alert.

Two kinds of failure never reach the dead letter queue: jobs blocked by the monthly request limit, and cron executions (crons have no retries and no dead letter queue).

Only the final attempt is listed under Dead letters. The earlier attempts stay in Jobs as Failed.

Find out why it failed

1. Go to Dead letters.
2. Narrow the list with Search by URL… or the All queues filter. The columns are URL, Queue, Attempts and Dead lettered.
3. Open the row’s … menu and click View details to see the last request, the response and the error.

Fix the cause before you replay, or the replay fails the same way. Debug a failed run explains each failure reason and what to check.

Replay jobs

Replay sends the request again as a new job:

• a new job_id, attempt 1, scheduled for now
• the same queue, URL, method, headers, payload and callback URL
• the queue’s current settings, including its full set of retries (on a paused queue, the job waits)

To replay one job, open its … menu and click Replay. To replay several, tick them and click Replay. To replay everything, click Replay all.

The dead-lettered entry stays in the list after a replay. Purge it once the replay has succeeded, so nobody replays it twice. Each replayed job counts toward your monthly requests and sends its own callback.

Caution

Replay, Replay all and Purge from a row’s menu run straight away, with no confirmation. Replay all and Purge all ignore the search box and the queue filter: they act on every dead-lettered job in the team.

Purge jobs

Purging deletes the dead-lettered job and its run record for good.

• One job: open its … menu and click Purge.
• Several jobs: tick them, click Purge and confirm in Purge selected jobs?.
• Everything: click Purge all and confirm in Purge all dead-lettered jobs?.

Automatic cleanup

Each queue’s Dead letter retention sets how long its dead-lettered jobs are kept: 3, 7, 14 (the default) or 30 days. Once a day at 00:00 UTC, Delay deletes the ones older than that. Change it on the queue’s edit page (Set up a queue). Deleting a queue deletes its dead-lettered jobs too.

The dead letter queue is in the dashboard only. There’s no API for it.

Next steps

• Jobs: how retries lead to the dead letter queue.
• Retries and queue design: retry delays that ride out an outage.
• Queues: the Dead letter retention setting and the other queue settings.