Chunking & Lazy Iteration

Process a million rows without blowing your memory limit — chunk, chunkById and cursor.

Model::all() is fine for a settings table and a disaster for an events table with ten million rows — it hydrates every record into memory at once. When you need to walk a large result set, you process it in batches.

The three approaches

chunk(n, ...) — runs a query per batch. Simple, but risky if you modify the rows you are paging over (you can skip records).
chunkById(n, ...) — pages by primary key instead of offset. Safe when the loop updates the rows. This is almost always the one you want.
cursor() / lazy() — a generator that keeps one model in memory at a time; lazy() even batches the underlying fetches.

User::where('active', true)->chunkById(500, function ($users) {
    foreach ($users as $user) {
        // heavy work, 500 at a time
    }
});

cursor() uses the least memory on the PHP side, but be aware it still buffers on the database driver by default and holds the connection open for the whole loop.

Example

<?php

namespace App\Jobs;

use App\Models\Subscription;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class ExpireStaleSubscriptions implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function handle(): void
    {
        // chunkById is SAFE while we mutate the rows we page over,
        // because it pages on the primary key, not a numeric offset.
        Subscription::query()
            ->where('status', 'active')
            ->where('ends_at', '<', now())
            ->chunkById(1000, function ($subscriptions) {
                foreach ($subscriptions as $subscription) {
                    $subscription->update(['status' => 'expired']);
                    \App\Jobs\NotifyExpiry::dispatch($subscription)->afterCommit();
                }
            });

        // For a pure read (e.g. exporting), cursor() keeps memory flat:
        // foreach (Subscription::where('status','expired')->cursor() as $sub) { ... }
    }
}

When to use it

A weekly email campaign iterates over 200,000 subscriber records using chunk(500) to stay within the PHP memory limit while dispatching individual jobs.
A data export job uses lazy() to stream millions of rows through a generator without loading them all into memory at once.
A database cleanup task uses chunkById() to safely delete old log records in batches, avoiding the row-offset drift that plain chunk() can cause.

More examples

chunk() for memory-safe batch processing

chunk() fetches 500 users at a time and passes each batch to the closure, releasing the previous batch before fetching the next.

// Process 500 users at a time
User::where('subscribed', true)
    ->chunk(500, function ($users) {
        foreach ($users as $user) {
            dispatch(new SendNewsletterJob($user));
        }
    });
// Memory stays low regardless of total user count

chunkById() for safe deletes and updates

chunkById() pages by primary key instead of OFFSET, so deleting rows in one chunk does not cause records to be skipped in the next.

// Safe when rows may be deleted mid-iteration
Log::where('created_at', '<', now()->subYear())
    ->chunkById(1000, function ($logs) {
        foreach ($logs as $log) {
            $log->delete();
        }
    });

lazy() and lazyById() generators

lazy() returns a LazyCollection backed by multiple chunked queries, exposing a generator interface so only one model is in memory at a time.

// Yields models one at a time via a cursor
foreach (User::lazy() as $user) {
    ProcessUserJob::dispatch($user);
}

// lazyById() pages by ID for safe modification
foreach (Product::where('stock', 0)->lazyById() as $product) {
    $product->update(['is_active' => false]);
}

Discussion

Be the first to comment on this lesson.