Identify file and string character encodings (UTF-8, UTF-16, ISO-8859-1, Windows-1252, ASCII, Shift-JIS), inspect hidden Byte Order Marks (BOM), repair scrambled Mojibake characters in 1 click, and transcode charsets in real time with 100% in-browser client-side privacy on RiazHub.com.
Universal Text Encoding Detector & Charset Converter
Identify file & string character encodings, detect Byte Order Marks (BOM), repair garbled Mojibake artifacts, and transcode charsets with sub-millisecond precision.
| Encoding / Standard | Confidence Probability | BOM & Magic Byte Status | Byte Validity | Quick Actions |
|---|---|---|---|---|
| Enter text or upload a file to calculate real-time charset probability rankings. | ||||
📚 Deep Dive: Encodings, BOM Headers & Mojibake Explained
🎯 What is a Byte Order Mark (BOM) & Why Does UTF-8 BOM Break PHP / Shell Scripts? ▾
A Byte Order Mark (BOM) is a sequence of magic bytes at the very beginning of a stream (e.g. EF BB BF in UTF-8 or FF FE in UTF-16LE). In UTF-16 and UTF-32, the BOM is required to signal endianness (Little Endian vs. Big Endian).
However, in UTF-8, endianness does not exist. Including a UTF-8 BOM often causes critical software malfunctions:
- PHP "Headers Already Sent" Errors: PHP outputs the invisible 3-byte BOM before your script runs, prematurely triggering HTTP response headers and breaking session cookies or
header('Location: ...')redirects. - JSON Parsing Crashes: Strict JSON parsers (RFC 8259) reject inputs starting with BOM bytes as invalid syntax.
- Unix Shebang Failures: Shell scripts beginning with
#!/bin/bashfail if preceded by BOM magic bytes.
🧩 What Causes "Mojibake" Garbled Text & How Does RiazHub Auto-Repair It? ▾
Mojibake (文字化け) is the Japanese term for corrupted character rendering caused when binary data encoded in one character set is decoded using a completely different, incompatible character set.
The most ubiquitous real-world corruption is Double-UTF-8 / Windows-1252 mismatch:
- A UTF-8 string containing accented letters (e.g.
é, encoded in bytesC3 A9) is mistakenly read by legacy software as two separate Latin-1 / Windows-1252 single-byte characters (Ãand©). - The resulting scrambled characters (
é) are re-saved as UTF-8, compounding the corruption. - Our Repair Engine: Our client-side pipeline extracts the raw byte values of the garbled string, maps them back through Windows-1252/ISO-8859 code points, and re-interprets them through a strict UTF-8 stream to flawlessly recover original characters (e.g.,
é➔é,€➔€,مرØبا➔مرحبا).
🌐 ASCII (7-bit) vs. ISO-8859 (8-bit) vs. Variable-Length Unicode (UTF-8) ▾
Understanding the evolutionary hierarchy of text encodings:
- US-ASCII (7-bit): Defined in 1963, covers 128 characters (0–127) including basic English letters, digits, and control codes. Any byte with the 8th bit set (≥ 128) is invalid ASCII.
- ISO-8859 & Windows-1252 (8-bit): Extended ASCII to 256 code points (0–255) to support Western European accented glyphs. Because 256 spots were inadequate for all world languages, hundreds of incompatible code pages emerged (e.g. CP-1256 for Arabic, CP-1251 for Cyrillic, Shift-JIS for Japanese).
- UTF-8 (Variable 1–4 Bytes): The universal standard powering >98% of the modern web. Backward compatible with 7-bit ASCII (1 byte), while seamlessly encoding over 149,000 Unicode characters (including Arabic, Chinese, emojis, math symbols) across 1 to 4 bytes.
🔒 100% In-Browser Privacy Guarantee — Zero Server Transmission ▾
Your security and confidentiality are paramount on RiazHub.com. This utility operates strictly inside your browser's sandboxed JavaScript execution thread using the HTML5 FileReader, Uint8Array, and TextDecoder APIs.
No text payloads, proprietary code files, SQL dumps, or authentication tokens are ever sent over network sockets or saved on external servers. You can safely disconnect your internet connection and the entire detector will continue running at full speed offline.