PHP mysql charset utf8 problems

PHP mysql charset utf8 problems

  1. Blog
  2. 15 November 2026
  3. 4 min read

Stefan Bogdanescu

Founder & Senior Architect

PHP mysql charset utf8 problems

Maintaining a legacy codebase often means dealing with a database that isn't perfectly consistent. You might find yourself staring at a schema where some tables are utf8 and others are still stuck on latin1. When you attempt to convert everything to UTF-8 by altering one table, the results can be surprisingly frustrating: half the characters render correctly, while others appear as question marks or garbled text.

This fragmentation usually points to a mismatch in the character encoding chain rather than an issue with the database engine itself. MySQL supports multiple character sets (like latin1, utf8, and utf8mb4), but it relies on the correct configuration of the client, the connection, and the server to interpret data consistently. If any link in this chain is configured differently, you will encounter the exact symptoms described above.

The Connection Chain

The core issue often lies in how PHP connects to MySQL. Even if a table is defined as utf8 (or better yet, utf8mb4), the connection itself might be sending data as latin1. By default, older PHP configurations and some hosting environments use latin1 for the client connection unless explicitly told otherwise.

To fix this, you need to ensure that the character set is negotiated correctly at the moment of connection. You can achieve this using the mysql_set_charset() function or by setting the charset attribute in the connection options if you are using MySQLi or PDO.

Using the legacy mysql_* extension (which is deprecated but still common in older codebases), you would typically add:

mysql_set_charset('utf8', $link);

However, relying on latin1 defaults is a common pitfall. If your HTML meta tag declares UTF-8, but the PHP script connects to MySQL using latin1, special characters like smart quotes (e.g., ’) will be interpreted as single-byte values in Latin-1. When these bytes are sent to a UTF-8 table, they often fail to map to valid Unicode code points, resulting in those question marks you see on the screen.

Database Schema Consistency

While fixing the connection is crucial, the database schema itself must be uniform. Mixing latin1 and utf8 tables in the same database can lead to subtle bugs, especially if you are storing data that crosses boundaries (e.g., user-generated content containing quotes).

You should verify that all tables use utf8mb4 rather than just utf8. The standard utf8 character set in MySQL is a misnomer; it only supports 3-byte UTF-8 sequences. It cannot store the full Unicode range, including emojis or certain special characters like the curly quotes mentioned in your source material.

To convert tables and columns correctly, you must alter them to explicitly support the full character set:

ALTER TABLE your_table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER DATABASE your_database_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Note that converting an existing table might fail if the current data contains characters that cannot be represented in the new encoding, though utf8 to utf8mb4 is generally safe as it is a superset. Always run these commands after ensuring your connection is already set to UTF-8 to prevent partial reads during the conversion process.

HTML and Browser Expectations

Finally, ensure that your HTML headers are consistent. The meta tag <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> tells the browser to interpret the page as UTF-8. If you have switched your PHP output and database to UTF-8 but forgotten this tag (or if it is set incorrectly), the browser will try to guess the encoding, often defaulting to latin1, which causes the display issues.

If you are working on a project where the original developer used mixed encodings, the most robust fix is to standardize the entire stack:

  1. Set the MySQL connection charset to utf8mb4.
  2. Update all database objects to use utf8mb4.
  3. Ensure PHP output buffering does not strip headers.

A Note on Modern Development

If you are maintaining a legacy site but have the option to rewrite parts of it, consider moving away from the deprecated mysql_* extension entirely. Using MySQLi or PDO with explicit charset options provides more control and cleaner separation of concerns. For example, in modern Laravel applications, the database configuration file often handles this automatically:

'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',

This approach ensures that the framework enforces UTF-8 everywhere without requiring manual mysql_set_charset() calls in every script. For more details on managing database configurations in modern PHP stacks, resources like https://laravelcompany.com offer insights into best practices for handling data integrity and migrations in enterprise environments.

Our network