Unicode

You have probably opened an old file or visited a website only to see strange gibberish, broken question marks, or garbled symbols. This happened because computers, at their core, do not understand letters or words at all.

Computers only understand numbers made of zeros and ones. To display text on a screen, they need an agreed-upon lookup table that pairs characters with numbers—such as '1 equals A, 2 equals B.' This reference table is called a character set or code page.

The problem was that in the past, each country and even individual companies created their own separate numbering systems. The United States used ASCII for English, Western Europe used ISO-8859-1 (Latin-1), and Korea used EUC-KR for Hangul. When you sent a document to a computer using a different code table, the characters became hopelessly scrambled because each machine interpreted the numbers differently.

As computers spread worldwide and software had to handle multiple languages at once, this chaos turned into a massive headache. Developers had to reconfigure character rules for every single country and system.

In the late 1980s, engineers at companies like Apple and Xerox came up with an ambitious idea: unite all human writing systems into one giant master table. This became Unicode, meaning a 'unified code.' The Unicode Consortium was formed in 1991, and as the internet exploded, it quickly became the global standard.

Unicode gave almost every character on Earth a unique, non-overlapping code number. It covers the Latin alphabet, Hangul, Chinese characters, Arabic, Hindi, ancient Egyptian hieroglyphs, and modern smartphone emoji. Today, no matter who a computer communicates with across the globe, it instantly recognizes the correct character just by checking its number.

To be precise, Unicode is the catalog of assigned numbers (called code points), while the method used to actually store those numbers in a computer file is a separate step. This storage format is called an encoding, and the most widely used standard today is UTF-8.

UTF-8 is a clever system that adjusts the byte size of each character dynamically. Basic English letters and digits (the ASCII range) are stored compactly in just 1 byte. Characters with higher code numbers, such as Korean Hangul or Chinese characters, take up 3 bytes, while emoji take 4 bytes. The size depends not on how visually complex a character is, but on which numeric range its code falls into.

Here is an important point to remember: UTF-8 is backward-compatible only with legacy English ASCII. If you open an older document saved in a legacy local standard like EUC-KR or Shift JIS using UTF-8, the text will still show up as gibberish. Even though Unicode assigns a permanent unique ID to each letter, your system must use the matching encoding rule to read and display the file correctly.