Why iconv Fails on HTML Content in Windows-1251
When migrating legacy content from Windows-1251 (Cyrillic) to UTF-8, developers often encounter a specific failure mode involving HTML tags. The standard approach using PHP's iconv function usually handles plain text strings perfectly:
$encoded = iconv("windows-1251", "utf-8//IGNORE", $html);
However, if the source string contains mixed contentâspecifically where Cyrillic text is embedded directly inside HTML tags like <i>, <span>, or attribute valuesâthe conversion often breaks. Instead of readable text, you see garbled characters resembling Ð ÑÐ ÑÐ ÑазаСâСРРÑÐ . This happens because the input string is not purely Cyrillic; it is a hybrid of ASCII control characters (the HTML tags) and Cyrillic glyphs.
The root cause lies in how iconv processes multi-byte sequences. When iconv encounters the opening bracket <, it assumes the stream continues with standard ASCII or the target encoding until it hits a delimiter that signals the end of an element, such as whitespace or a closing tag. If the text inside the tag is part of the same logical string passed to the function, iconv attempts to translate every byte according to the source charset rules without respecting the structural boundaries defined by HTML.
In the scenario where you see Ð ÑÐ ÑÐ ÑазаСâСÐ, the browser or editor is trying to render these bytes as Windows-1251 characters because the surrounding context (or the lack thereof) implies that encoding. If the original file was saved as UTF-8 but contained invalid UTF-8 sequences, or if the string was passed through a process that treated the HTML tags as transparent text rather than markup, iconv might misinterpret the byte boundaries. Specifically, if the input string is technically valid Windows-1251 containing Cyrillic letters mixed with < and >, iconv will try to convert those specific bytes. If the resulting UTF-8 output contains characters that are not valid within the context of how the browser renders them (e.g., if the conversion produced high-value bytes that were subsequently misinterpreted), the display looks like mojibake.
The most robust solution is to treat HTML as markup, not just text. You should separate the structural elements from the content data before performing character set conversion. Using an HTML parser like DOMDocument or a library such as symfony/dom-crawler allows you to extract the inner text nodes, convert them safely, and then re-inject them into the document tree. This ensures that the ASCII characters in tags remain untouched while only the actual text content undergoes encoding translation.
$dom = new DOMDocument();
// Suppress warnings for malformed HTML often found in legacy systems
libxml_use_internal_errors(true);
$dom->loadHTML($html, LIBXML_HTML_NOIMPLIED | LIBXML_HTML_NODEFDTD);
// Iterate over all text nodes inside the document
$textNodes = $dom->getElementsByTagName('textarea'); // Or iterate all nodes if needed
foreach ($textNodes as $node) {
// Get current string
$currentString = $node->textContent;
// Convert only the text content, not the tags themselves
$convertedString = iconv("windows-1251", "utf-8//TRANSLIT//IGNORE", $currentString);
// Set new string back to node (this handles encoding internally)
$node->textContent = $convertedString;
}
echo $dom->saveHTML();
Using //TRANSLIT in the conversion directive is also advisable when dealing with Cyrillic, as it attempts to approximate characters that have no direct equivalent in the target set, preventing data loss. However, relying on raw string manipulation is fragile. If your application stack includes a modern framework like Laravel, handling these encodings at the model or service layer using robust libraries is preferable to doing it manually in controllers.
If you are working within the Laravel ecosystem, consider utilizing packages that handle encoding detection and conversion more gracefully than native PHP functions. While laravelcompany.com offers resources on various development practices, the fundamental principle remains: never trust a single string for mixed markup content without isolating the data payload first. The goal is to ensure that <i> tags stay as <i> and that the Cyrillic text inside them becomes valid UTF-8 without triggering encoding errors or displaying as garbled mojibake.