Streaming Large Files in PHP Without Loading Them into Memory
Serving files larger than a few megabytes from non-web-accessible directories on shared hosting requires a specific approach to memory management. The default behavior of readfile() is to load the entire file into RAM before transmitting it to the client. For a 200MB file, this risks exhausting the memory limit (memory_limit directive) enforced by shared hosting providers, potentially crashing the script or causing the server to reject the request.
The solution lies in stream processing. Instead of reading the whole file at once, you read it in chunks and flush the output buffer after each chunk. This keeps memory usage constant regardless of file size. However, simply opening a file with fopen() and looping through it is not enough; you must also handle HTTP headers correctly to ensure the browser treats the response as a file download rather than plain text.
Setting Up Headers Correctly
Before reading any bytes from the disk, you must send the necessary HTTP headers. These inform the browser about the content type, file size, and disposition. It is crucial to set these headers before echoing or flushing any output.
$fullname = "/path/to/restricted/uploads/document.pdf"; // Non-web-accessible path
if (auth_check()) {
$fd = fopen($fullname, "rb");
if ($fd) {
$fsize = filesize($fullname);
$parts = pathinfo($fullname);
// Determine MIME type based on extension
switch ($parts["extension"]) {
case "pdf": header("Content-type: application/pdf"); break;
case "zip": header("Content-type: application/zip"); break;
default: header("Content-type: application/octet-stream"); break;
}
// Essential for triggering a download
header("Content-Disposition: attachment; filename=\"{$parts['basename']}\"");
// Inform the client of the total size
header("Content-length: $fsize");
// Prevent caching to ensure fresh content on every request
header("Cache-control: private");
echo "Stream started";
ob_flush();
flush(); // Must flush before any output is sent
} else {
http_response_code(404);
exit("Error opening file");
}
fclose($fd);
}
Implementing Chunked Reading
The core of the streaming logic involves reading a specific byte count from the file descriptor rather than using readfile(). A common mistake is using while(!feof($fd)) without checking if data was actually read. This can lead to an infinite loop if the stream reaches EOF unexpectedly or due to buffering issues.
Instead, use fread() with a fixed buffer size. Reading 1MB chunks strikes a good balance between minimizing system calls and keeping memory overhead low. After reading, you must echo the buffer and flush the output buffer immediately using ob_flush() and flush(). This ensures the data leaves the server as soon as it is read, rather than waiting for the entire file to be processed or the script to terminate.
while (!feof($fd)) {
$buffer = fread($fd, 1 * 1024 * 1024); // Read 1MB chunks
if ($buffer === false) {
break; // Stop if reading fails
}
echo $buffer;
ob_flush();
flush();
}
Handling Shared Hosting Constraints
On shared hosting environments, you often lack control over Apache modules like X-Sendfile, which allow web servers to serve files from arbitrary locations without loading them into PHP memory. Without such modules, PHP must handle the streaming entirely. Additionally, some hosts restrict the maximum execution time (max_execution_time) or memory usage.
Streaming mitigates memory issues but does not automatically solve timeout problems if the script takes too long. For a 200MB file streamed at 1MB per chunk with a flush, the loop runs roughly 200 times. If each iteration takes even a fraction of a second, you might approach time limits. However, since I/O is usually the bottleneck, streaming is generally faster than loading into memory. If timeouts occur, increasing max_execution_time or using an asynchronous queue system might be necessary, though shared hosting often prohibits background processing for downloads.
Security Considerations
Keeping files outside the web root is a valid security practice to prevent direct URL access. However, ensure that the file path construction $path . $name is validated strictly against a whitelist of allowed filenames to prevent directory traversal attacks (e.g., ../../etc/passwd). Never trust user input or database values directly as file paths without sanitization.
By combining proper HTTP headers with chunked reading and flushing, you can serve large files securely from restricted directories without hitting memory limits on shared hosting.