How to Avoid Maximum Execution Time Exceeded Without Changing php.ini
When exporting large datasetsâlike 100,000 recordsâfrom a Laravel application, hitting the max_execution_time limit is a common frustration. The standard advice is often to edit php.ini, but that requires server access and doesn't scale well in shared hosting or containerized environments. The goal is to keep the script running within the default 60-second window by optimizing how data flows through your application.
The issue isn't just about fetching data; it's about what happens immediately after you fetch it. In your specific case, the bottleneck lies in loading the entire result set into memory before processing it in chunks.
The Memory vs. Time Trade-off
Your current implementation uses AssetRepository::query()->get(). Even though you intend to use chunk() later, calling ->get() first retrieves all 100,000 rows from the PostgreSQL database and loads them into a single collection in memory.
This creates two problems:
- Memory Spikes: Loading 100k rows into an array before processing can spike memory usage significantly, potentially triggering OOM (Out Of Memory) errors or causing PHP to swap heavily.
- Execution Time Cost: The time spent fetching and hydrating all objects happens synchronously in the main request cycle. If your
max_execution_timeis 60 seconds, you have limited headroom for the subsequent logic.
To stay within the 60-second limit without changing server configuration, you must offload the heavy lifting to the database layer itself. This means leveraging SQL cursor-based processing rather than loading everything into PHP first.
Implementing Database-Level Chunking
Laravel's DB::cursor() is the key to solving this without modifying php.ini. Unlike chunk(), which still requires fetching a batch of results before yielding them (though it does so efficiently), cursor() returns a generator that fetches rows from the database one by one as they are requested.
Here is how you should refactor the controller logic:
public function exportAll(Request $request)
{
$data = AssetRepository::query();
$headers = [
'Content-Type' => 'text/csv',
'Cache-Control' => 'must-revalidate, post-check=0, pre-check=0',
'Content-Disposition' => 'attachment; filename=export.csv',
'Expires' => '0',
'Pragma' => 'public',
];
$response = new StreamedResponse(function () use ($data) {
$handle = fopen('php://output', 'w');
// Load static lookups once (these are small enough to fit in memory)
$remark = Remark::all(['id','label','type']);
$remarkAsset = RemarkAsset::all(['asset_id','value','remark_id']);
// Build a lookup map for remarks to avoid repeated array searches
$remarkMap = collect($remark)->keyBy('type');
$assetRemarkMap = collect($remarkAsset)->keyBy('asset_id');
// Initialize the CSV header
$headersRow = ['id', 'duration', 'qr_code', 'material_no', /* ... all other columns */];
fputcsv($handle, $headersRow);
// Use cursor() instead of get() to fetch rows lazily
foreach ($data->query()->cursor() as $asset) {
$row = [];
// Map static remark headers
foreach ($remarkMap->keys() as $type) {
$value = '';
if ($remarkMap->has($type)) {
// Logic to find the specific remark value based on asset attributes
// This part needs to be optimized further if it involves complex lookups per row
// For now, assuming simple mapping or empty string if not found
}
$row[$type] = $value;
}
// Add static asset columns
$row['id'] = $asset->id;
$row['duration'] = $asset->duration;
// ... add