Retries, Failures and Idempotency

Attempts, backoff, the failed_jobs table and queue:retry, plus why every handler must survive running twice.

Syntaxpublic function failed(Throwable $exception): void

An attempt is one delivery of a job to a worker. If handle() throws, the job goes back on the queue and is attempted again. When it runs out of attempts it is recorded as failed and never retried on its own.

Setting the limits

Laravel 13 lets you declare the limits as attributes on the class, or as methods and properties on it. The two forms are the same thing, and the attribute version exists so the values are visible at the top of the file.

#[Tries(5)]      is  public function tries(): int { return 5; }
#[Backoff(10)]   is  public function backoff(): int|array { return 10; }
#[Timeout(30)]   is  public $timeout = 30;

Note the odd one out: there is no timeout() method. Tries and Backoff each fall back to a method of the same name, while Timeout falls back only to a $timeout property. Note too that Backoff takes its seconds as a plain argument, not as a named delay: argument.

An array backoff gives a growing delay: [10, 60, 300] waits ten seconds, then a minute, then five minutes and five minutes for any further attempt. retryUntil() returns a DateTime and replaces counting with a deadline. Zero tries means unlimited, so pair it with #[MaxExceptions].

When attempts run out

The failed job is written to the failed_jobs table with a uuid, the payload and the exception. php artisan queue:failed lists them, queue:retry {uuid} or queue:retry all puts them back, queue:forget {uuid} drops one, queue:flush drops all and queue:prune-failed --hours=48 trims old ones. Create the table with php artisan make:queue-failed-table if it is missing.

Assume it runs twice

A queue is at-least-once. A worker killed after the work but before deleting the job, or a job that overruns retry_after, is delivered again. Make handle() check whether the work is already done before doing it.

Example

Example · php
<?php
// app/Jobs/ChargeInvoice.php

declare(strict_types=1);

namespace App\Jobs;

use App\Models\Invoice;
use App\Services\PaymentGateway;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Attributes\Backoff;
use Illuminate\Queue\Attributes\MaxExceptions;
use Illuminate\Queue\Attributes\Timeout;
use Illuminate\Queue\Attributes\Tries;
use Throwable;

#[Tries(5)]
#[Timeout(30)]
#[MaxExceptions(3)]
#[Backoff([10, 60, 300])]
class ChargeInvoice implements ShouldQueue
{
    use Queueable;

    public function __construct(
        public Invoice $invoice,
    ) {}

    public function handle(PaymentGateway $gateway): void
    {
        // 1. Already done? A second delivery stops here.
        if ($this->invoice->paid_at !== null) {
            return;
        }

        // 2. A key the gateway can deduplicate on, derived from the
        //    invoice rather than from this attempt.
        $charge = $gateway->charge(
            amountInCents: $this->invoice->total_cents,
            idempotencyKey: "invoice-{$this->invoice->id}",
        );

        // 3. Record the outcome so step 1 can see it next time.
        $this->invoice->update([
            'paid_at' => now(),
            'charge_id' => $charge->id,
        ]);
    }

    /**
     * Called once, after the final attempt fails.
     */
    public function failed(Throwable $exception): void
    {
        $this->invoice->update([
            'payment_error' => $exception->getMessage(),
        ]);
    }
}

When to use it

  • Keep a flaky third-party call retrying with a growing delay instead of failing on the first timeout.
  • Notify an admin from failed() when a payment job gives up.
  • Re-run yesterday's failed media jobs after fixing the bug that broke them.
  • Add a unique index so a job delivered twice cannot create two rows.

More examples

The method forms, and a deadline instead of a count

Use the method forms when a value is computed rather than literal, for example a backoff that depends on the job's own data. retryUntil suits a job whose work stops being useful after a while: a catalogue sync that is half an hour late should be abandoned, not retried for ever. Note that retryUntil is evaluated once, at dispatch, while the payload is being built, and the timestamp it returns is then carried in that payload for every later attempt.

Example · php
<?php
// app/Jobs/SyncSupplierCatalogue.php

declare(strict_types=1);

namespace App\Jobs;

use DateTime;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class SyncSupplierCatalogue implements ShouldQueue
{
    use Queueable;

    // Same as #[Tries(0)]: unlimited attempts.
    public function tries(): int
    {
        return 0;
    }

    // Same as #[Backoff([30, 120, 600])].
    public function backoff(): int|array
    {
        return [30, 120, 600];
    }

    // Unlimited attempts, but stop trying after 30 minutes.
    public function retryUntil(): DateTime
    {
        return now()->addMinutes(30);
    }

    public function handle(): void
    {
        // ...
    }
}

Working the failed queue

queue:retry does not run the job; it moves the stored payload back onto the queue for a worker to pick up, so a worker must be running for anything to happen. The payload is the one captured at the original dispatch, which means a retry re-fetches models by key and sees today's rows, not the rows from when it first failed.

Example · shell
# What failed, when, and with which exception.
php artisan queue:failed

# Put one back on its original queue, by uuid.
php artisan queue:retry 9a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9

# Everything, or just one queue's worth.
php artisan queue:retry all
php artisan queue:retry --queue=media

# Delete one, or all of them.
php artisan queue:forget 9a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9
php artisan queue:flush

# Housekeeping: drop records older than two days.
php artisan queue:prune-failed --hours=48

# If the table does not exist yet.
php artisan make:queue-failed-table
php artisan migrate

What idempotency looks like without the framework

Run the same call twice and the total is unchanged. That is the only property that matters: the handler's effect must depend on the job's data, not on how many times it ran. In a real application the has() check is a unique index on an idempotency column, or firstOrCreate, or updateOrCreate, so two workers racing cannot both pass the check.

Example · php
<?php

// Pretend this is the charges table, keyed by the key the
// handler derives from the invoice.
$charges = collect([
    'invoice-41' => 2500,
]);

$apply = function (Illuminate\Support\Collection $charges, int $invoiceId, int $cents) {
    $key = "invoice-{$invoiceId}";

    // The whole trick: a stable key, and a check before the write.
    return $charges->has($key)
        ? $charges
        : $charges->put($key, $cents);
};

$charges = $apply($charges, 41, 2500);   // second delivery, ignored
$charges = $apply($charges, 42, 900);    // first delivery, applied

echo $charges->sum();
// Output:
// 3400

Related

laravel jobslaravel queue driverslaravel job patternslaravel horizonlaravel adv transactions

Discussion

  • Be the first to comment on this lesson.