// character encoding

What Is UTF-8?

UTF-8 is a character encoding used to represent text inside computers. It provides a way of turning Unicode characters such as letters, numbers, mathematical symbols and characters from many languages into bytes that a computer can store and process.

// definition

UTF-8 turns characters into bytes

UTF-8 stands for Unicode Transformation Format - 8-bit.

It is an encoding system that converts Unicode characters into sequences of bytes.

This allows text to be stored in files, transmitted across networks and interpreted consistently by different computer systems.

// why encoding?

Computers do not store letters directly

When we look at a computer screen, we see characters:

A £ π é 日 😀

But computer storage ultimately consists of binary data. The computer therefore needs a defined system for representing each character using numbers and bytes.

Character → Unicode code point → UTF-8 encoding → Bytes

// unicode

First, understand Unicode

Unicode is a standard that assigns characters unique numerical identifiers called code points.

A Unicode code point is normally written using U+ followed by hexadecimal digits.

Character	Unicode code point
A	U+0041
£	U+00A3
é	U+00E9
π	U+03C0
€	U+20AC
😀	U+1F600

// unicode vs utf-8

Unicode and UTF-8 are not the same thing

This distinction is important.

Unicode

Defines characters and assigns them code points.

UTF-8

Defines how those Unicode code points are encoded into bytes.

Unicode tells us that the pound sign is U+00A3. UTF-8 tells the computer which bytes should be used to represent that character.

// bytes

UTF-8 uses between one and four bytes

UTF-8 is a variable-length encoding.

Different characters require different numbers of bytes.

Character	UTF-8 bytes	Bytes used
A	41	1
£	C2 A3	2
π	CF 80	2
€	E2 82 AC	3
😀	F0 9F 98 80	4

Important: the hexadecimal byte values shown above are the UTF-8 encoding of each character, not the Unicode code point itself.

// ascii compatibility

UTF-8 is compatible with ASCII

One of the reasons UTF-8 became so successful is its compatibility with ASCII.

The first 128 Unicode characters use exactly the same single-byte values as ASCII.

A = U+0041 = UTF-8 byte 41 B = U+0042 = UTF-8 byte 42 1 = U+0031 = UTF-8 byte 31

This means ordinary English text using basic letters, digits and punctuation is very compact in UTF-8.

// multilingual text

UTF-8 can represent text from many languages

UTF-8 is not limited to English.

English: Hello Português: Olá Español: Información Français: Éducation Greek: π Japanese: 日本 Currency: £ € ¥ Mathematics: ≤ ≥ ± √ ∞ Emoji: 😀

All of these characters can exist within the same UTF-8 encoded document.

// encoding errors

What happens when the wrong encoding is used?

Text can become corrupted-looking when software interprets bytes using the wrong character encoding.

For example, a pound sign encoded as UTF-8 may sometimes incorrectly appear as:

Expected: £50 Incorrect interpretation: £50

This type of garbled text is often called mojibake.

The data may not actually be damaged. The same bytes may simply have been interpreted using the wrong encoding.

// txt files

UTF-8 and .txt files

A .txt file tells us that a file contains plain text, but it does not automatically tell us which character encoding was used.

.txt ≠ automatically UTF-8

A text file might use UTF-8, ASCII, Windows-1252 or another encoding.

For modern text files, UTF-8 is generally the most useful and interoperable choice.

// csv files

UTF-8 and CSV

CSV is also a text-based format, so character encoding matters.

Name,City,Amount João,Lisboa,£50 Élodie,Paris,€75 Sofia,Athens,€40

Saving this file using UTF-8 helps preserve accented characters and currency symbols when the file moves between compatible systems.

// xml

UTF-8 and XML

XML documents can explicitly declare their character encoding.

<?xml version="1.0" encoding="UTF-8"?>

This tells the software reading the document that its characters are encoded using UTF-8.

// html

UTF-8 on websites

Websites also need to specify how text should be interpreted.

Modern HTML pages commonly include:

<meta charset="UTF-8">

This tells the browser to interpret the document using UTF-8.

Correct encoding is important for displaying names, symbols, mathematical notation, international text and other characters reliably.

// advantages

Why is UTF-8 so widely used?

Unicode support

It can represent the full range of Unicode code points.

ASCII compatible

Basic ASCII characters retain their original one-byte representation.

Efficient

Common English characters require only one byte, while additional bytes are used when necessary.

Portable

UTF-8 is supported across operating systems, programming languages, databases and web technology.

// where it appears

UTF-8 appears throughout modern computing

UTF-8 can be used with many text-based technologies and file formats.

TXT CSV XML JSON HTML CSS JavaScript APIs Databases Web Pages

// key idea

The file format and the encoding are different things

This is one of the most useful concepts to understand.

File format: describes how information is organised.

Character encoding: describes how the characters themselves are represented as bytes.

CSV = structure of the data UTF-8 = encoding of the characters

A CSV file can therefore be encoded using UTF-8. An XML document can be encoded using UTF-8. A plain .txt file can also be encoded using UTF-8.

// summary

Characters become bytes.

UTF-8 is a character encoding that converts Unicode characters into sequences of one to four bytes. It allows computers to store and exchange text from many languages, symbols and writing systems while remaining compatible with ASCII.