How to remove duplicate values from a multi-dimensional array in PHP

How to remove duplicate values from a multi-dimensional array in PHP

  1. Blog
  2. 15 November 2026
  3. 4 min read

Stefan Bogdanescu

Founder & Senior Architect

Duplicating Arrays: A Deep Dive into Deduplication Strategies

When you encounter a multi-dimensional array where entire sub-arrays are repeated, the solution isn't as simple as applying array_unique(). That built-in function operates on one dimension at a time and compares values using loose comparison by default. If your data structure looks like a list of records, removing duplicates often hinges on whether you want to treat two arrays as identical if their values match exactly, or if the order of elements within those sub-arrays matters.

Consider an array where index 0 and index 3 contain structurally identical data:

Array (
    [0] => Array ([0] => abc, [1] => def)
    [3] => Array ([0] => abc, [1] => def)
)

A naive approach might try to serialize each sub-array into a string and then filter. While serialize() works for complex objects, it can produce unpredictable output for simple arrays depending on PHP version nuances. A more robust method involves converting each sub-array into a hashable format. Since the example shows strictly numeric keys starting from zero, you can safely use json_encode() to create a canonical string representation of each inner array.

$data = [
    ['abc', 'def'],
    ['ghi', 'jkl'],
    ['mno', 'pql'],
    ['abc', 'def'], // Duplicate
];

$seen = [];
$uniqueData = [];

foreach ($data as $item) {
    $hash = json_encode($item);
    
    if (!isset($seen[$hash])) {
        $seen[$hash] = true;
        $uniqueData[] = $item;
    }
}

This pattern iterates through the collection, encodes each sub-array into a JSON string, and checks if that specific signature has been encountered. The resulting $uniqueData will preserve the original order of first occurrence while eliminating subsequent duplicates. This is generally preferred over array_unique() because it maintains stability; array_unique() can reorder elements or alter keys depending on whether you use strict comparison (SORT_STRING) and how your data types are mixed.

Handling Associative Keys and Order Sensitivity

The logic above assumes that the sub-arrays only have numeric keys starting from zero, as seen in your example. However, real-world data often involves associative arrays where key names carry semantic meaning. If you have an array like ['id' => 1, 'name' => 'John'], sorting or reordering these keys during comparison could break the logic if not handled carefully.

json_encode() handles this gracefully by preserving key order and name in the resulting string (prior to PHP 5.4, it sorted keys; in modern versions, it respects insertion order). This means that [0] => abc is distinct from abc => [1], preventing accidental merges of structurally different data.

However, there is a trade-off: JSON encoding increases memory usage for large datasets because you are storing string representations alongside the original values. If performance becomes a bottleneck with millions of records, consider implementing a custom hashing function that skips encoding but strictly compares numeric keys and values in a single pass. Alternatively, if you are working within the Laravel ecosystem, you might find that helpers or collections make this iteration more readable:

$unique = collect($data)->unique()->values()->toArray();

While collect() provides a fluent interface, be aware that ->unique() on multi-dimensional arrays relies on the same underlying serialization logic. It is convenient but less performant than a raw loop for massive datasets. Always benchmark against your specific data volume before choosing a collection helper over a manual loop.

Why Not Just Use array_unique()?

You might wonder why array_unique() isn't the first recommendation. The core issue lies in how it treats multi-dimensional arrays. It does not perform a deep comparison by default. If you pass a 2D array to array_unique(), it will likely throw an error or, depending on PHP version and flags, only process the top-level keys. Even with SORT_REGULAR, it compares values element-by-element but cannot reliably determine if two distinct sub-arrays are duplicates unless they appear in a flattened context or specific strict mode is enforced.

Attempting to force array_unique() often leads to unexpected results where elements are removed that should remain, simply because the comparison logic doesn't account for the nested structure you intend to deduplicate. The serialization approach guarantees that two arrays are only considered identical if every key-value pair matches exactly in the same order.

Final Considerations on Data Integrity

When removing duplicates, the biggest pitfall is assuming that value equality implies structural identity. In your example, ['abc', 'def'] and ['abc', 'def'] are clearly duplicates. But what if one is an array of strings and the other contains integers that look identical? json_encode() distinguishes between "abc" and 123 correctly, whereas loose comparison might not.

If you need to preserve original keys after deduplication rather than resetting them to a sequential 0, 1, 2 index, ensure you are appending to the result array directly without calling reset() or re-indexing. The manual loop approach naturally preserves the indices of the first occurrence of each unique sub-array.

For complex projects involving large-scale data cleaning, this pattern remains one of the most reliable ways to enforce uniqueness on nested structures without relying on fragile built-in behaviors that change between PHP versions.

Our network