Fixing Broken UTF-8 Encoding in MySQL
When you see characters like ̨̮ or ÃÆÃ appearing in your database, it is almost certainly a mojibake issue caused by a mismatch between the character set actually stored on disk and the character set your application expects. This specific patternâwhere a single accented character turns into two (or sometimes three) standard ASCII charactersâis the hallmark of an ISO-8859-1 or Windows-1252 byte stream being interpreted as UTF-8.
In your specific scenario, you are running PHP 5 with a proper UTF-8 header, likely displaying data from a MySQL column defined as utf8_general_ci. The fact that Notepad++ reads the file as UTF-8 without BOM suggests the source files are clean, but the database content itself remains corrupted. Since not all accented characters are broken, this confirms that the corruption is legacy data rather than a configuration error in your current stack.
To resolve this, you need a strategy that maps these double-encoded bytes back to their intended single-byte representations before converting them to valid UTF-8. You cannot simply run a CONVERT statement on the entire column if the underlying byte values are already invalid UTF-8 sequences; MySQL will either truncate the data or report an error depending on its configuration.
The most robust approach is to create a conversion function that handles the specific "double-byte" corruption manually. The character ̨̮ is actually the result of the byte 0xC3 (part of UTF-8) being misinterpreted as Latin-1, followed by 0x82, which is then also misinterpreted. By treating the stored string as ISO-8859-1 and decoding it first, you recover the original bytes that were intended to be UTF-8.
Here is a PHP function designed for this exact restoration process:
function fixEncoding($input) {
// Step 1: Treat the broken string as ISO-8859-1 to get the raw bytes
$decoded = iconv('ISO-8859-1', 'UTF-8//IGNORE', $input);
// Step 2: The result is now valid UTF-8, but we need to clean up any
// remaining invalid sequences if they existed.
return $decoded;
}
When using this function, you must apply it row-by-row or batch-process your table, as MySQL does not allow direct updates on columns containing invalid UTF-8 sequences without triggering errors in strict modes. You can wrap this logic in a script that iterates through the affected rows and applies UPDATE statements conditionally.
However, there is a critical trade-off to consider regarding data integrity. If your database was originally created with Latin-1 encoding but you have since switched your application stack to UTF-8 without cleaning the old data, simply converting the character set of the table (ALTER TABLE ... CONVERT TO CHARACTER SET utf8) will fail immediately because the bytes are invalid. You must fix the logical content first, then change the physical storage format.
Once the data is logically corrected using the conversion function, you can safely alter the column definition:
ALTER TABLE your_table_name
MODIFY COLUMN your_column_name VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci;
After this step, your application will handle new inserts correctly. It is also worth noting that while utf8_general_ci was the standard for a long time, modern applications often benefit from utf8mb4. If you decide to migrate later, ensure you have sufficient disk space, as utf8mb4 uses up to 4 bytes per character instead of 3.
If you are using Laravel, this manual cleanup phase is still necessary if the issue already exists in the database; schema migrations alone cannot fix existing binary data corruption. Once the legacy mojibake is cleared, your application will behave consistently regardless of the client tool used to view the data.
