Unicode 与中文：为什么 \u4e2d 是"中"，以及 Emoji 代理对的长度坑Unicode and Chinese: Why \u4e2d Means "中", and the Emoji Surrogate Pair Length Trap
从 `\u4e2d` 到"中"只差一次码点查表，但 Emoji 一出现，`string.length` 就开始撒谎。本文讲清码点、编码单元与代理对的关系，以及我在前后端联调中踩过的三个真实坑From `\u4e2d` to "中" is just one code-point lookup, but the moment Emoji enters the picture, `string.length` starts lying. This article explains code points, code units and surrogate pairs, plus three real bugs I hit during frontend-backend integration.
从 \u4e2d 到"中"：码点才是 Unicode 的原子From \u4e2d to "中": the code point is Unicode\'s atom
很多人第一次看到 `\u4e2d` 是在 JSON 里——接口返回了一串 `\u4e2d\u6587`，前端渲染出来却是"中文"。这不是加密，而是 Unicode 转义：`\u` 后面跟 4 位十六进制数，表示一个码点（code point）。`4e2d` 换算成十进制是 19990，去 Unicode 码表里查，第 19990 号字符就是"中"。Most people first encounter `\u4e2d` inside JSON — an API returns `\u4e2d\u6587` and the frontend renders "中文". This is not encryption; it is a Unicode escape: `\u` followed by four hex digits denotes one code point. `4e2d` is 19990 in decimal, and character number 19990 in the Unicode table is "中".
码点是 Unicode 给每个字符分配的唯一编号，范围从 U+0000 到 U+10FFFF，共一百多万个。ASCII 只覆盖前 128 个，中文基本区在 U+4E00–U+9FFF，Emoji 和生僻字则分布在 U+10000 以上的"增补平面"。理解码点是理解一切编码问题的起点：`\u4e2d` 不是"中"的某种加密形式，它就是"中"的编号而已。A code point is the unique number Unicode assigns to every character, ranging from U+0000 to U+10FFFF — over a million slots. ASCII covers the first 128, the CJK Unified Ideographs block sits at U+4E00–U+9FFF, and Emoji plus rare characters live in the "supplementary planes" above U+10000. Grasping code points is the starting point for every encoding problem: `\u4e2d` is not some encrypted form of "中", it is simply its index number.
UTF-8 与 UTF-16：编码单元不等于码点UTF-8 and UTF-16: code units are not code points
码点是抽象编号，落到存储和传输里必须编码成字节。UTF-8 用 1–4 个字节表示一个码点，ASCII 范围内的字符只占 1 字节，中文通常占 3 字节，Emoji 占 4 字节。这也是为什么"中文"两个字在 UTF-8 里是 6 个字节，而不是 2 个。Code points are abstract numbers; in storage and transmission they must be encoded into bytes. UTF-8 uses 1–4 bytes per code point: ASCII characters take one byte, Chinese usually takes three, and Emoji takes four. That is why "中文" is six bytes in UTF-8, not two.
UTF-16 则用 2 个字节作为一个编码单元（code unit）。U+FFFF 以内的码点一个单元就够了，但 U+10000 以上的码点必须拆成两个单元——这就是代理对（surrogate pair）。高代理在 U+D800–U+DBFF，低代理在 U+DC00–U+FFFF，两者配对才能还原出真正的码点。JavaScript 的字符串内部就是 UTF-16，所以 `"😀".length` 返回 2 而不是 1：它数的是编码单元，不是码点。我曾经在做昵称长度校验时直接用 `str.length > 20`，结果用户输入一个 Emoji 就被多算了一位，后台数据库按字符数存又没问题，前后端校验对不上，排查了半天才定位到这个差异。UTF-16 uses two bytes as one code unit. Code points up to U+FFFF fit in a single unit, but anything above U+10000 must be split into two units — a surrogate pair. The high surrogate ranges from U+D800 to U+DBFF and the low surrogate from U+DC00 to U+FFFF; only together do they reconstruct the real code point. JavaScript strings are internally UTF-16, which is why `"😀".length` returns 2 instead of 1: it counts code units, not code points. I once wrote a nickname length check as `str.length > 20`, and a user typing one Emoji got counted as two characters while the backend stored by code point with no issue — the mismatch took me half a day to track down.
Emoji 的组合字符：长度问题远不止代理对Emoji combining sequences: the length problem goes beyond surrogate pairs
如果说代理对只是"一个字符占两个单元"，那组合字符（combining character）和零宽连接符（ZWJ）才是真正的噩梦。`"👨👩👧"` 这个家庭 Emoji 看起来是一个字符，实际上由 5 个码点组成：男人、ZWJ、女人、ZWJ、女孩，在 JS 里 `.length` 是 8（5 个码点中有 3 个需要代理对，共 8 个编码单元）。肤色修饰符、国旗的区域指示符对，都是类似的多码点序列。If surrogate pairs are merely "one character in two units", combining characters and the Zero Width Joiner (ZWJ) are the real nightmare. The family Emoji `"👨👩👧"` looks like one character but is actually five code points: man, ZWJ, woman, ZWJ, girl — and its `.length` in JavaScript is 8 (three of the five code points need surrogate pairs, totalling eight code units). Skin-tone modifiers and regional-indicator pairs for flags follow the same multi-code-point pattern.
我踩过的另一个坑是截断：做摘要预览时按前 30 个编码单元截断，结果把一个代理对从中间砍断，渲染出乱码方块。正确做法是按码点迭代（ES2015 的 `Array.from(str)` 或 `for...of`），或者用 `Intl.Segmenter` 按用户感知的字素簇（grapheme cluster）切分。数据库层面也要注意：MySQL 的 `utf8mb3` 存不了 Emoji，必须用 `utf8mb4`，否则写入直接报错或静默截断。这些问题的根源都是同一个——把"编码单元"当成了"字符"。Another bug I hit was truncation: a preview snippet cut at the first 30 code units sliced a surrogate pair in half, rendering a broken tofu box. The fix is to iterate by code point (`Array.from(str)` or `for...of` in ES2015), or use `Intl.Segmenter` to split by user-perceived grapheme clusters. On the database side, MySQL's `utf8mb3` cannot store Emoji — you need `utf8mb4`, or writes fail or silently truncate. All these problems share one root cause: treating code units as characters.
实战排查：用工具把不可见的结构可视化Practical debugging: visualising the invisible structure
遇到乱码或长度对不上时，我的排查流程是固定的三步：先确认源数据的字节序列，再确认每一层（浏览器、接口、数据库）用的编码，最后把可疑字符串转成码点序列看结构。比如用户反馈"昵称显示正常但搜索不到"，往往是因为输入时用了全角空格或兼容字符，看起来一样但码点不同。When I hit mojibake or length mismatches, my debugging routine is three fixed steps: confirm the raw byte sequence of the source data, verify the encoding used at every layer (browser, API, database), then convert the suspicious string into a code-point sequence to inspect its structure. For example, when a user reports "my nickname displays fine but search can't find it", the cause is often a full-width space or compatibility character that looks identical but has a different code point.
本站的 Unicode 工具可以在文本和 `\uXXXX` 转义之间互转，支持中文、Emoji 和代理对的正确解析，把一段字符串贴进去就能看到每个字符对应的码点和 UTF-8 字节。排查编码问题时，能"看见"结构比凭经验猜要快得多。Our Unicode tool converts between text and `\uXXXX` escapes, correctly handling Chinese, Emoji and surrogate pairs — paste a string in and you see every character's code point and UTF-8 bytes. When debugging encoding issues, being able to "see" the structure is far faster than guessing from experience.