Counting Characters — Bytes vs. Code Points vs. Graphemes

"How many characters is this?" sounds simple, but there are several ways to count. With non-English text or emoji, tools and languages can disagree. The reason is three different units of counting: bytes, code points, and graphemes. Let’s look at each.

Bytes: the storage and transfer size

A byte is the unit of size when a computer stores or sends data. When text becomes bytes, UTF-8 is the widely used scheme. In UTF-8, ASCII letters and digits are 1 byte, while many CJK characters are 3 bytes. Database column limits and payload sizes are often measured in bytes, not characters.

Code points: Unicode’s numbers

Unicode assigns every character a unique number (code point). "A" is one code point, but emoji and combined characters can be made of several code points. A program’s string length is often counted in these units, which is one reason it diverges from the visible character count.

Graphemes: what a person sees as one character

A grapheme (grapheme cluster) is what a person perceives as a single character. For example, a family emoji 👨‍👩‍👧 is built from multiple code points but looks like one character, so it counts as one grapheme. Combined accented characters behave the same way. This is the count closest to human intuition.

UTF-8 and Shift_JIS

Two encodings that come up often with Japanese are UTF-8 and Shift_JIS. UTF-8 is the modern standard that can handle characters from around the world and is the default for the web and recent systems (1–4 bytes per character; most Japanese characters are about 3). Shift_JIS is an older scheme long used in Japanese environments and still lingers in some CSV files and legacy systems. The same text produces completely different byte sequences.

Comparing bytes per character

Even the same "one character" takes a different number of bytes depending on the encoding. Counting representative characters in UTF-8, Shift_JIS, and UTF-16 gives the following (measured on this site).

• A (ASCII letter) — UTF-8: 1 / Shift_JIS: 1 / UTF-16: 2 bytes
• é (accented Latin) — UTF-8: 2 / Shift_JIS: cannot represent / UTF-16: 2 bytes
• あ (hiragana) — UTF-8: 3 / Shift_JIS: 2 / UTF-16: 2 bytes
• 漢 (kanji) — UTF-8: 3 / Shift_JIS: 2 / UTF-16: 2 bytes
• 😀 (emoji) — UTF-8: 4 / Shift_JIS: cannot represent / UTF-16: 4 bytes (surrogate pair)

Two points stand out. Japanese characters take 3 bytes in UTF-8 but 2 in Shift_JIS, and emoji or accented characters cannot be represented in Shift_JIS at all. Where a form or database limit is measured in bytes, this difference applies directly.

Why does mojibake (garbled text) happen?

Garbled text happens when the encoding used to save differs from the one used to read. For example, opening a UTF-8 file as Shift_JIS turns Japanese into meaningless symbols. The classic "opened a CSV in Excel and it was garbled" problem is usually this. Prevent it by standardizing files on UTF-8, or by specifying the correct encoding when opening.

Why the numbers differ

• "あ" → 1 grapheme / 1 code point / 3 bytes (UTF-8)
• Emoji 😀 → 1 grapheme / 1 code point / 2 UTF-16 units (surrogate pair)
• Family emoji 👨‍👩‍👧 → 1 grapheme, but multiple code points
• Whether a social limit or form cap counts in which unit depends on the spec

Our character counter shows graphemes, code points, UTF-16 units, and UTF-8 bytes together, so you can find the "right" count for your purpose. Understanding it alongside Base64 (carrying text as bytes) and URL encoding makes the relationship between characters and bytes click.