Embedding

Imagine a gym class where you line up your classmates on the playground. There's just one rule: stand near people who like the same things you do. The kids who love soccer gather on one side, and the kids who love drawing gather on the other.

Once everyone is in place, something neat happens. You don't have to ask who is similar to whom. You can just measure the distance between two people. You don't even need to know their names. Where they stand tells you how they relate.

We play the same game when we teach a computer about words. A computer doesn't know what the letters in 'apple' mean. You might think you could just label apple as number 1 and lion as number 2. But that doesn't work. The fact that 1 and 2 are neighbors has nothing to do with what the two words mean.

So instead, each word gets a set of numbers that works like its spot on the playground. These coordinates are called an embedding.

When the spots on the playground are arranged well, direction starts to mean something too, not just distance. Draw an arrow from where 'man' stands to where 'king' stands. Then start at 'woman' and walk the same direction for the same length.

You end up near 'queen.' That's why people say 'king minus man plus woman equals queen.' It works because the arrow carries the idea of royal rank.

Thanks to this, a computer can do quite a lot without knowing what any word means. Type 'cheap laptop' into a search box, and it also finds 'inexpensive laptop.' The two phrases stand close together.

Recommendation features run on the same idea. They pick other videos that sit near the one you just watched. Calculating which spots are nearby is the same thing as finding similar items.

People don't assign these spots one by one. The computer reads a huge amount of text and adjusts the arrangement on its own. It nudges words closer together when they show up with similar neighbors (their context), and it repeats that nudge over and over. 'Cheap' and 'inexpensive' rarely appear in the same sentence. But both often show up next to words like 'laptop' and 'price,' so they drift close together.

The playground is also much bigger than we tend to picture. It doesn't have just two directions, left-right and up-down. It stretches in hundreds or even thousands of directions. So the coordinates are made of hundreds or thousands of numbers. But no single number comes with a label like 'sweetness' or 'size.'

The math doesn't always work out neatly, either. If you follow the arrow, you only land near 'queen,' not exactly on her spot. And many word pairs don't work this way at all.

If the text the computer read is skewed, the arrangement gets tilted too. Biases in what people have written seep into the coordinates, and developers are working to spot and fix that.