Laravel Queues and Asynchronous Tasks

Q: What is the difference between queue:work and queue:listen ?

queue:work runs in memory without restarting the framework (fast, but requires a manual restart after code changes); queue:listen restarts the framework for each task (slow, but automatically loads new code). Use queue:work + Supervisor in production, and use queue:listen or queue:work --once during development.

Q: Can I use Eloquent models in a Job?

Yes, the SerializesModels trait automatically serializes model IDs and retrieves them from the database during deserialization. The benefit is that Jobs have a smaller data footprint; the risk is that if a model is deleted, the Job will fail (this can be ignored using deleteWhenMissingModels ).

Q: How do I debug a job?

During development, use QUEUE_CONNECTION=sync to run the job synchronously (errors are displayed immediately); in production, use php artisan queue:failed to view error messages for failed jobs; you can also log detailed information in the job's failed() method.

Queues are Laravel's "background team"—time-consuming tasks are offloaded to the queue for asynchronous processing, so users don't have to wait and requests are returned instantly.

1. What You'll Learn

Queue-Driven Configuration: Comparison of sync/database/redis/sqs
Job Creation and Dispatch: dispatch() / Queue::push()
Failure retry mechanism: the tries/backoff/failed_jobs table
Queue Priority, Latency, and Batch Tasks: Batch/Bused
Supervisor monitors the queue:work process

2. A User's Story

(1) Pain Point: Exporting reports causes the entire system to freeze

Bob clicked "Export Monthly Report" in the ShopMetrics admin panel—it took 30 seconds to generate a CSV file containing 100,000 order records, during which time the page kept spinning, and Alice also experienced slow performance when accessing the dashboard at the same time. To make matters worse, Bob triggered five report exports simultaneously, exhausting the PHP process pool and causing the entire site to return a 502 error.

(2) Solution Using Queues

The queue sends time-consuming tasks to the background—when a user clicks "Export," they immediately see a "Report is being generated" message, while the job runs slowly in the background; once it's finished, an email is sent with a download link.

PHP

// Sync — blocks for 30 seconds
$csv = ReportService::generateMonthlyReport($tenant);

// Async — returns instantly, Job runs in background
GenerateReportJob::dispatch($tenant, 'monthly');
// User sees: "Report is being generated. We'll email you when it's ready."

(3) Revenue

After Bob implemented the queue, report export requests now return within 100 ms, background jobs run smoothly, and Alice's dashboard no longer freezes.

3. Queue-Driven Configuration

(1) Driver Comparison

Driver	Persistence	Performance	Suitability	Cost
`sync`	❌ Instant Execution	Fastest	Development/Testing	Free
`database`	✅ DB table	Slow	Small project	Free
`redis`	✅ Memory	Fast	Production Environment	Redis
`sqs`	✅ AWS	High	Large-scale	Pay-as-you-go
`beanstalkd`	✅ Dedicated	Fast	Medium	Free

(2) Configuration

BASH

# .env
QUEUE_CONNECTION=redis

# Redis connection
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

PHP

// config/queue.php
'connections' => [
    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
    ],
    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => '{default}',
        'retry_after' => 90,
        'block_for' => null,
    ],
],

(3) Create a Team List

BASH

# For database driver
php artisan queue:table
php artisan queue:failed-table
php artisan migrate

# Creates: jobs table + failed_jobs table

(1) ▶ Example: ShopMetrics Queue Configuration

BASH

# .env — Development
QUEUE_CONNECTION=database

# .env — Production
QUEUE_CONNECTION=redis
REDIS_HOST=redis.shopmetrics.internal
REDIS_PORT=6379

# Create queue tables (database driver)
php artisan queue:table
php artisan queue:failed-table
php artisan queue:batches-table
php artisan migrate

Output:

TEXT

# Command executed successfully

4. Job Creation and Distribution

(1) Create a Job

BASH

php artisan make:job GenerateReportJob
# Creates: app/Jobs/GenerateReportJob.php

(2) Define a Job

PHP

// app/Jobs/GenerateReportJob.php
class GenerateReportJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public int $backoff = 60;
    public bool $deleteWhenMissingModels = true;

    public function __construct(
        public Tenant $tenant,
        public string $reportType,
        public string $format = 'csv',
    ) {}

    public function handle(
        ReportService $reportService,
        Mailer $mailer,
    ): void {
        $path = $reportService->generate(
            $this->tenant,
            $this->reportType,
            $this->format,
        );

        $url = Storage::disk('s3')->temporaryUrl($path, now()->addDays(7));

        $this->tenant->users->each(function ($user) use ($url) {
            $mailer->to($user)->send(new ReportReadyNotification($url, $this->reportType));
        });
    }

    public function failed(\Throwable $exception): void
    {
        Log::error('Report generation failed', [
            'tenant_id' => $this->tenant->id,
            'report_type' => $this->reportType,
            'error' => $exception->getMessage(),
        ]);
    }
}

(3) Distribute the Job

PHP

// Basic dispatch
GenerateReportJob::dispatch($tenant, 'monthly');

// Delayed dispatch
GenerateReportJob::dispatch($tenant, 'monthly')
    ->delay(now()->addMinutes(5));

// Dispatch to specific queue
GenerateReportJob::dispatch($tenant, 'monthly')
    ->onQueue('reports');

// Dispatch if condition met
GenerateReportJob::dispatchIf($tenant->subscription?->isActive(), $tenant, 'monthly');

// Dispatch after response sent to user
GenerateReportJob::dispatchAfterResponse($tenant, 'monthly');

Distribution Method	Description	Applicable Scenarios
`dispatch()`	Enqueue	General Asynchronous Task
`dispatchSync()`	Execute synchronously	Must be completed immediately
`dispatchAfterResponse()`	Execute after response	Lightweight task
`delay()`	Delayed Execution	Scheduled Tasks
`onQueue()`	Specified Queue	By Priority

(1) ▶ Example: ShopMetrics Order Processing Job

PHP

// app/Jobs/ProcessOrderJob.php
class ProcessOrderJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public int $backoff = [30, 60, 120];

    public function __construct(public Order $order) {}

    public function handle(
        OrderService $orderService,
        PaymentGateway $payment,
    ): void {
        // Charge payment
        $payment->charge($this->order);

        // Update inventory
        $orderService->deductInventory($this->order);

        // Broadcast event
        event(new OrderPlaced($this->order));

        // Send confirmation email
        $this->order->user->notify(new OrderConfirmationNotification($this->order));
    }

    public function failed(\Throwable $exception): void
    {
        $this->order->update(['status' => 'failed']);
        $this->order->user->notify(new OrderFailedNotification($this->order));
    }
}

Output:

TEXT

// Execution Successful

5. Failure Retry Mechanism

(1) Retry Configuration

PHP

// In Job class
public int $tries = 3;           // Max retry attempts
public int $backoff = 60;        // Seconds between retries
public int $timeout = 120;       // Max seconds per attempt

// Or exponential backoff
public array $backoff = [30, 60, 120]; // 30s, 60s, 120s

// Or dynamic backoff
public function backoff(): int
{
    return 30 * $this->attempts();
}

(2) Error Handling

BASH

# View failed jobs
php artisan queue:failed

# Retry a specific failed job
php artisan queue:retry 5

# Retry all failed jobs
php artisan queue:retry all

# Delete a failed job
php artisan queue:forget 5

# Clear all failed jobs
php artisan queue:flush

(3) Queue Task Lifecycle

flowchart LR
    A[Job Dispatched] --> B[Queue]
    B --> C[Worker Picks Up]
    C --> D{Success?}
    D -->|Yes| E[Job Completed]
    D -->|No| F{attempts < tries?}
    F -->|Yes| G[Backoff + Retry]
    G --> B
    F -->|No| H[failed_jobs Table]
    H --> I[Manual Retry / Flush]

(1) ▶ Example: ShopMetrics Failure Retry Configuration

PHP

// app/Jobs/SendWebhookJob.php
class SendWebhookJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 5;
    public array $backoff = [10, 30, 60, 120, 300];
    public int $timeout = 30;

    public function __construct(
        public Shop $shop,
        public array $payload,
    ) {}

    public function handle(): void
    {
        $response = Http::timeout($this->timeout)
            ->post($this->shop->webhook_url, $this->payload);

        if (!$response->successful()) {
            $this->release($this->backoff[$this->attempts() - 1] ?? 60);
        }
    }

    public function failed(\Throwable $exception): void
    {
        $this->shop->tenant->users->each(function ($user) {
            $user->notify(new WebhookFailedNotification($this->shop));
        });
    }
}

Output:

TEXT

// Execution Successful

6. Batch Tasks

(1) Create a Batch

BASH

php artisan queue:batches-table
php artisan migrate

PHP

// app/Jobs/ProcessTenantAnalyticsJob.php
class ProcessTenantAnalyticsJob implements ShouldQueue
{
    use Batchable;

    public function __construct(public Tenant $tenant) {}

    public function handle(): void
    {
        if ($this->batch()->cancelled()) {
            return;
        }

        AnalyticsService::computeForTenant($this->tenant);
    }
}

(2) Distribute Batch

PHP

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch(
    Tenant::active()->get()->map(
        fn ($tenant) => new ProcessTenantAnalyticsJob($tenant)
    )
)->then(function (Batch $batch) {
    // All jobs completed successfully
    Log::info("Batch {$batch->id} completed: {$batch->totalJobs} tenants processed.");
})->catch(function (Batch $batch, \Throwable $e) {
    // First job failure
    Log::error("Batch {$batch->id} failed: {$e->getMessage()}");
})->finally(function (Batch $batch) {
    // Always runs (success or failure)
    Cache::forget('analytics:computing');
})->name('Process Monthly Analytics')
  ->onQueue('analytics')
  ->dispatch();

(3) Batch Management

PHP

// Check batch progress
$batch = Bus::findBatch($batchId);
$batch->progress();     // 0-100
$batch->processedJobs();
$batch->totalJobs();
$batch->failedJobs();
$batch->finished();

// Cancel batch
$batch->cancel();

Method	Description
`then()`	All callbacks successful
`catch()`	First-attempt failure callback
`finally()`	Callback upon completion (regardless of success or failure)
`progress()`	Percentage Complete
`cancel()`	Cancel Remaining Tasks

(1) ▶ Example: ShopMetrics Monthly Analysis Batch Task

PHP

// app/Console/Commands/ProcessMonthlyAnalytics.php
class ProcessMonthlyAnalytics extends Command
{
    protected $signature = 'analytics:process-monthly';

    public function handle(): int
    {
        $tenants = Tenant::active()->get();
        $this->info("Processing analytics for {$tenants->count()} tenants...");

        Bus::batch(
            $tenants->map(fn ($t) => new ProcessTenantAnalyticsJob($t))
        )->then(function (Batch $batch) {
            $this->info("All {$batch->totalJobs} tenants processed.");
        })->catch(function (Batch $batch, \Throwable $e) {
            $this->error("Batch failed: {$e->getMessage()}");
        })->name('Monthly Analytics')
          ->onQueue('analytics')
          ->allowFailures()
          ->dispatch();

        return self::SUCCESS;
    }
}

Output:

TEXT

// Execution Successful

7. Supervisor Daemon

(1) Install Supervisor

BASH

# Ubuntu/Debian
sudo apt-get install supervisor

# Create config
sudo nano /etc/supervisor/conf.d/shopmetrics-worker.conf

(2) Supervisor Configuration

INI

[program:shopmetrics-worker-default]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/shopmetrics/artisan queue:work redis --queue=default --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/var/log/shopmetrics/worker-default.log
stopwaitsecs=3600

[program:shopmetrics-worker-reports]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/shopmetrics/artisan queue:work redis --queue=reports --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
numprocs=1
user=www-data
redirect_stderr=true
stdout_logfile=/var/log/shopmetrics/worker-reports.log

(3) The `supervisor` Command

BASH

# Read new config
sudo supervisorctl reread
sudo supervisorctl update

# Start/stop/restart workers
sudo supervisorctl start shopmetrics-worker-default:*
sudo supervisorctl stop shopmetrics-worker-default:*
sudo supervisorctl restart shopmetrics-worker-default:*

# Check status
sudo supervisorctl status

Number of Processes	Queue	CPU	Memory	Description
2	default	Low	Medium	General Tasks
1	reports	High	High	Report Generation
1	analytics	High	High	Data Analysis
1	notifications	Low	Low	Email/Push

(1) ▶ Example: Starting a ShopMetrics Multi-Queue Worker

BASH

# Development — single worker, all queues
php artisan queue:work --queue=default,reports,notifications

# Production — separate workers per queue priority
# Priority: high > default > low
php artisan queue:work redis --queue=high,default
php artisan queue:work redis --queue=reports,low

# Process jobs with time limit (auto-restart for memory leaks)
php artisan queue:work --max-time=3600 --max-jobs=1000

# Monitor queue
php artisan queue:monitor redis:default,redis:reports

Output:

TEXT

# Command executed successfully

8. Comprehensive Example: ShopMetrics Asynchronous Reporting System

PHP

// ============================================
// Comprehensive: ShopMetrics Async Report System
// Covers: jobs, batches, retries, supervisor, notifications
// ============================================

// app/Jobs/GenerateReportJob.php
class GenerateReportJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public array $backoff = [60, 180, 600];
    public int $timeout = 600;
    public bool $deleteWhenMissingModels = true;

    public function __construct(
        public Tenant $tenant,
        public string $reportType,
        public string $format = 'csv',
        public ?int $userId = null,
    ) {
        $this->onQueue('reports');
    }

    public function handle(ReportService $reportService): void
    {
        $path = $reportService->generate(
            $this->tenant,
            $this->reportType,
            $this->format,
        );

        $downloadUrl = Storage::disk('s3')->temporaryUrl($path, now()->addDays(7));

        $user = $this->userId ? User::find($this->userId) : $this->tenant->users->first();
        $user?->notify(new ReportReadyNotification(
            downloadUrl: $downloadUrl,
            reportType: $this->reportType,
            expiresAt: now()->addDays(7),
        ));
    }

    public function failed(\Throwable $exception): void
    {
        $user = $this->userId ? User::find($this->userId) : $this->tenant->users->first();
        $user?->notify(new ReportFailedNotification(
            reportType: $this->reportType,
            error: $exception->getMessage(),
        ));
    }
}

// Usage in controller
class ReportController extends Controller
{
    public function generate(Request $request): JsonResponse
    {
        $validated = $request->validate([
            'report_type' => 'required|in:monthly,weekly,custom',
            'format' => 'sometimes|in:csv,xlsx,pdf',
            'date_from' => 'sometimes|date',
            'date_to' => 'sometimes|date|after:date_from',
        ]);

        GenerateReportJob::dispatch(
            tenant(),
            $validated['report_type'],
            $validated['format'] ?? 'csv',
            auth()->id(),
        );

        return response()->json([
            'message' => 'Report generation started. You will receive an email when ready.',
            'estimated_time' => '5-15 minutes',
        ], 202);
    }
}

❓ FAQ

Q What is the difference between queue:work and queue:listen?

A queue:work runs in memory without restarting the framework (fast, but requires a manual restart after code changes); queue:listen restarts the framework for each task (slow, but automatically loads new code). Use queue:work + Supervisor in production, and use queue:listen or queue:work --once during development.

Q Can I use Eloquent models in a Job?

A Yes, the SerializesModels trait automatically serializes model IDs and retrieves them from the database during deserialization. The benefit is that Jobs have a smaller data footprint; the risk is that if a model is deleted, the Job will fail (this can be ignored using deleteWhenMissingModels).

Q How can I prevent the queue from backing up?

A Increase the number of workers (numprocs), use a higher-performance driver (Redis), set a job timeout to prevent deadlocks, and set up alerts to monitor the queue length.

Q What should I do if one task in a batch fails?

A By default, Batch cancels the remaining tasks upon encountering the first failure. Set allowFailures() to allow the batch to continue executing despite partial failures. Use Bus::findBatch() to view failure details and manually retry the failed tasks.

Q How do I debug a job?

A During development, use QUEUE_CONNECTION=sync to run the job synchronously (errors are displayed immediately); in production, use php artisan queue:failed to view error messages for failed jobs; you can also log detailed information in the job's failed() method.

Q What is the difference between Supervisor and systemd?

A Supervisor is a Python process manager that is easy to configure and is officially recommended by Laravel; systemd is Linux's native service manager, which offers better performance but is more complex to configure. Both can run the queue:work process as a daemon.

📖 Summary

Queues make time-consuming tasks asynchronous, allowing user requests to be responded to within seconds
Redis is the best queue driver for production environments, offering high performance and persistence.
The Job uses the tries/backoff configuration to set the retry strategy; exponential backoff prevents an avalanche.
Batch processing of large numbers of similar tasks, with support for progress tracking and partial failures
The Supervisor monitors Worker processes and automatically restarts them if they crash.
Multiple queues with priority levels: high/default/reports/low

📝 Exercises

Basic Exercise (⭐): Create a GenerateReportJob, dispatch an asynchronous request to generate a CSV report in the controller, test using the database driver, and run queue:work to verify that the job executes.
Advanced Exercise (⭐⭐): Configure the Redis queue driver to implement 3 retries with exponential backoff (30s/60s/120s), send notifications to users when a job fails, and test manual retries (queue:retry) .
Challenge (⭐⭐⭐): Use Bus::batch() to implement a batch task for monthly multi-tenant analysis—iterate through all active tenants, create one job per tenant, monitor progress percentage, clear the cache upon completion, and configure Supervisor to monitor the worker.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

404 Not Found

Laravel Queues and Asynchronous Tasks

1. What You'll Learn

2. A User's Story

(1) Pain Point: Exporting reports causes the entire system to freeze

(2) Solution Using Queues

(3) Revenue

3. Queue-Driven Configuration

(1) Driver Comparison

(2) Configuration

(3) Create a Team List

(1) ▶ Example: ShopMetrics Queue Configuration

4. Job Creation and Distribution

(1) Create a Job

(2) Define a Job

(3) Distribute the Job

(1) ▶ Example: ShopMetrics Order Processing Job

5. Failure Retry Mechanism

(1) Retry Configuration

(2) Error Handling

(3) Queue Task Lifecycle

(1) ▶ Example: ShopMetrics Failure Retry Configuration

6. Batch Tasks

(1) Create a Batch

(2) Distribute Batch

(3) Batch Management

(1) ▶ Example: ShopMetrics Monthly Analysis Batch Task

7. Supervisor Daemon

(1) Install Supervisor

(2) Supervisor Configuration

(3) The supervisor Command

(1) ▶ Example: Starting a ShopMetrics Multi-Queue Worker

8. Comprehensive Example: ShopMetrics Asynchronous Reporting System

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好

(3) The `supervisor` Command