PHP decoding and encoding json with unicode characters

PHP decoding and encoding json with unicode characters

  1. Blog
  2. 15 November 2026
  3. 4 min read

Stefan Bogdanescu

Founder & Senior Architect

Decoding and Encoding JSON with Unicode Characters: A Practical Guide

Handling Unicode characters in JSON streams often triggers confusion because the data appears intact in the raw stream but corrupts during the decode/encode cycle. The core issue usually stems from a mismatch between how the string is stored in memory, how json_decode interprets it, and how json_encode outputs it back.

The JSON specification explicitly states that strings can contain any Unicode character except quotes, backslashes, or control characters. When you receive a payload like {"Tag":"Odómetro"}, the bytes arriving over the network are typically UTF-8. If your application environment (such as legacy PHP 5.3) expects ISO-8859-1 (Latin-1), the raw UTF-8 bytes for 'ó' (which is two bytes: 0xC3 0xB3) are interpreted incorrectly as four separate Latin-1 characters, resulting in the garbled output Odómetro.

Attempting to fix this with utf8_encode() in PHP is a common pitfall. This function assumes the input string is ISO-8859-1 and converts it to UTF-8. If your input is already valid UTF-8 or if the encoding assumptions of the environment are different, applying utf8_encode will double-encode the bytes, permanently mangling the character. You must ensure the internal representation of the string matches the encoding expected by the JSON decoder before you touch it.

The Double-Encoding Trap

When you observe a result like [Tag] => Odómetro, the string is technically valid UTF-8, but it represents the text Odó instead of Odó. This happens because the original ó (UTF-8: C3 B3) was misinterpreted as two ISO-8859-1 characters (à and ´). When you then try to re-encode this corrupted string, it remains a valid UTF-8 sequence of bytes, but semantically incorrect.

The solution is not to force an encoding conversion unless you are certain the source data is actually Latin-1. If your application runs in an environment where the default internal encoding is UTF-8 (which is standard for modern PHP setups), simply decoding the JSON without modification usually preserves the characters correctly:

$jsonString = '{"Tag":"Odómetro"}';
$data = json_decode($jsonString, true);
print_r($data); // Outputs: Array ( [Tag] => Odómetro )

In this scenario, json_decode reads the UTF-8 bytes and reconstructs the Unicode character 'ó' correctly in memory. The corruption only occurs if the incoming string was already mangled or if the environment's internal encoding is misconfigured.

Handling Escaped Sequences

A frequent source of friction arises when json_encode escapes non-ASCII characters by default. For example, encoding the array containing Odómetro might produce:

{"Tag":"Od\u00f3metro"}

This behavior is correct according to the JSON spec and ensures that the output is ASCII-safe. However, if your downstream system requires literal Unicode characters instead of escape sequences, you need to configure the encoder accordingly.

In modern PHP versions (5.4+), the JSON_UNESCAPED_UNICODE flag solves this:

$json = json_encode($array, JSON_UNESCAPED_UNICODE);
// Outputs: {"Tag":"Odómetro"}

If you are constrained to older environments like PHP 5.3, this flag is unavailable. The source material notes an attempt to use it resulted in a type error warning because the function signature or availability changed between versions. In such legacy cases, relying on regex to post-process the output is fragile and not recommended. Instead, focus on ensuring the input data is clean UTF-8 before decoding.

Ensuring Input Integrity

The most robust approach is to guarantee that the input JSON string is valid UTF-8 before passing it to json_decode. If you are receiving data from an external source that claims to be UTF-8 but contains mojibake (like ó), you must detect and fix the encoding mismatch first.

A reliable way to handle this without assuming the wrong input encoding is to validate the byte order mark or inspect the raw bytes. However, in many production scenarios involving legacy integrations, the issue is simply that the server's default internal encoding was not UTF-8. Setting the locale to a UTF-8 aware one (e.g., setlocale(LC_ALL, 'en_US.UTF-8')) can help, but it does not fix already corrupted strings in memory.

If you encounter the specific case where json_encode escapes characters and you cannot upgrade PHP to use JSON_UNESCAPED_UNICODE, the only viable path for PHP 5.3 is manual string manipulation. You would need to replace \uXXXX sequences with their corresponding UTF-8 bytes or characters, but this requires a comprehensive regex that covers all edge cases, including surrogate pairs for emojis. Given the complexity and fragility of such regex patterns, upgrading the runtime environment is almost always the superior engineering decision.

Why This Matters in Production

When building APIs or processing data pipelines, consistency is key. If your application decodes JSON and then immediately re-encodes it to cache or log it, you must decide on a standard: ASCII-safe escape sequences or literal Unicode. Mixing these two strategies leads to data loss when characters are round-tripped through systems with different expectations.

By understanding that json_decode expects UTF-8 by default in most modern contexts and that json_encode defaults to escaping non-ASCII, you can write code that handles the transition gracefully. Avoid using functions like utf8_encode unless you have confirmed the input is strictly Latin-1. Instead, trust the JSON decoder to handle valid UTF-8 inputs and use configuration flags on the encoder to control the output format based on your specific requirements.

Our network