How can I clear the memory while running a long PHP script? tried unset()
You are facing a classic symptom of in-memory processing: Fatal error: Allowed memory size exhausted. You have already taken the standard first stepsâraising the memory_limit to 812M, setting set_time_limit(0), and attempting to unset() variables inside your loop. However, none of this resolves the crash. The specific error message indicates you tried to allocate just 71 bytes after hitting a hard cap, suggesting that while individual allocations might be small, the aggregate heap usage has saturated the process.
The core issue in your provided script stems from how Magento's application context is initialized and how memory is managed within the Zend engine. You are currently running this entire import sequence as a single, monolithic block of code inside the global scope. Crucially, you have instantiated Mage::app() at the very beginning:
require_once(CfgMagentoPath . '/app/Mage.php');
Mage::app();
This call initializes the Magento framework globally. Once Mage::app() is invoked, it loads classes, initializes models, and prepares the application context. In PHP, when memory runs low, the garbage collector (GC) does run, but it operates on a generation-based system. It collects unreachable objects, but if your global scope holds references to massive objects that the GC cannot easily identify as disposable without a manual trigger, or if you are holding onto large arrays like $CM and $IT, memory usage will only grow until the limit is hit.
Simply unset()ing variables inside a loop often fails in this context because those variables might be referenced internally by other objects loaded by Magento, or the GC hasn't had the chance to run yet. The most effective architectural change is not just wrapping the loop in a function, but restructuring the entire execution flow to isolate the heavy lifting into small, discrete units of work.
The Function Isolation Strategy
You asked if creating the main for loop inside a function would help. The answer is yes, but with a specific condition: you must instantiate the Magento application inside that function, not globally.
If you move the foreach($IT as $ItemRow) block into a function called processBatch(), and you do this within a tight loop (e.g., processing 100 rows at a time), you force PHP to destroy the entire function scope after each iteration. When the function ends, every variable defined within itâincluding Magento models, sessions, and temporary arraysâbecomes unreachable and is eligible for immediate garbage collection before the next batch begins.
Here is how that logic should look:
// Instead of one massive loop
foreach($IT as $ItemRow) { ... }
// Break into batches or smaller chunks
$batchSize = 50;
for ($i = 0; $i < count($IT); $i += $batchSize) {
$batch = array_slice($IT, $i, $batchSize);
// Initialize Magento only for this batch
require_once(CfgMagentoPath . '/app/Mage.php');
Mage::app();
foreach ($batch as $ItemRow) {
// Your logic here
// ...
// Cleanup specific variables if needed,
// though scope exit handles most of it.
}
// Explicitly clear the batch data to ensure GC can run
unset($batch);
// Reindex or flush caches here if necessary, then close app
Mage_Reindex();
}
Handling Large Arrays and Excel Data
Your script loads the entire Excel file into memory using $excel->getWorksheetData('Sheet1'). This is often the culprit in import scripts. If you are reading a file with thousands of rows, storing it all in an array like $IT creates a massive object graph that persists for the lifetime of the script.
A more robust approach is to iterate over the Excel reader directly if the underlying library supports streaming, or to manually slice the data before passing it to your processing logic. Even better, avoid loading the Category Map ($CM) entirely into memory if possible. If CatMap_Search requires the full array, consider implementing a database-backed lookup for categories instead of keeping the entire map in RAM.
Additionally, check your error handling arrays: $AAnotmapped, $AAnewproducts, etc. While you commented them out at the bottom, if they are being populated during execution, ensure they aren't growing indefinitely. If you need to track errors, consider writing them to a temporary file instead of keeping them in memory.
Final Configuration Checks
If you cannot refactor the code immediately due to legacy constraints, there is one final configuration step often overlooked: gc_enable(). Ensure that garbage collection is active. You can explicitly trigger it between batches using gc_collect_cycles(). However, relying solely on this is less effective than restructuring the scope as described above.
By isolating the Magento application lifecycle to small windows of time and memory, you allow the Zend engine to reclaim the heap effectively between each batch of products, preventing the cumulative growth that leads to the fatal error.