Vector representation

When you look at a photo of a dog, you just know it's a dog. You don't think about it. A computer can't do that. It has no eyes and no sense of what a dog is. The only thing it can work with is numbers.

So before a computer can do anything with a photo, a sentence, or a click, that thing has to become numbers. That part is easy. A photo is already a grid of pixels, and each pixel has a brightness. A sentence is made of letters, and each letter has a code.

The hard part is choosing which numbers. Some numbers help the computer, and some tell it almost nothing.

Why not just give everything a number?

Let's try the simplest idea first. Give each thing an ID:

• Dog = 1
• Puppy = 2
• Car = 3

The computer can now tell these three apart. But look at what's missing. Is a puppy more like a dog or more like a car? The numbers can't say. We could just as easily have made car number 2. The IDs are labels, and labels carry no meaning.

That's a real problem, because the question we need computers to answer, over and over, is "Is this like that?" Is this new photo like the dog photos? Does this sentence mean the same as that one? If the numbers can't express likeness, the computer has no way to recognize something it hasn't seen before.

What would useful numbers look like?

Instead of one label, let's describe each thing with a few traits. Say we score everything from 0 to 10 on two questions: How big is it? and How alive is it?

Now each thing is a short list of numbers, and those numbers mean something. A short list of numbers like this is called a vector.

How do we measure "alike"?

Here's the useful part. A list of two numbers is a point you can draw. Put size on one axis and alive on the other, and mark each thing on the page.

The puppy lands near the dog. The car lands far away from both. Without any extra rules, the picture already shows what we wanted: things that are alike end up close together. "How alike are these?" has become "How far apart are these points?", and distance is something a computer can calculate easily.

Sometimes we care less about the distance between two points and more about whether they point the same way from the center of the page. That measure is called cosine similarity. It's close to 1 when two vectors point the same way and close to 0 when they're unrelated. It's the same idea of likeness, measured a slightly different way.

But real things have more than two traits, right?

Right. Two traits are enough to draw on a page, but they can't capture everything about a dog, a sentence, or a song. Real systems use hundreds or thousands of numbers per item. We can't draw a thousand axes, but the idea doesn't change: similar things sit close together, and different things sit far apart.

There's one more big difference. In our example, we chose the traits. In real systems, nobody does. The computer works out its own numbers by studying huge amounts of data, and it places things so that what's similar ends up nearby. These learned vectors are called embeddings. Their individual numbers usually don't match neat human traits like "size," but together they capture likeness better than anything we could design by hand.

Can these numbers capture more than likeness?

Yes, and this is where vectors get interesting. If you subtract one vector from another, you get the direction from one point to the other. Directions can carry meaning too.

The famous example uses word embeddings. The direction from "man" to "woman" turns out to be roughly the same as the direction from "king" to "queen." So if you take king, subtract man, and add woman, you land near queen. It doesn't work perfectly or for every pair of words, but it works often enough to show that relationships, and not only likeness, are stored in the geometry.

So, how does a computer see data?

As points in space. Every photo, sentence, or click becomes a vector, a list of numbers that places it somewhere. When the numbers are chosen well, the space itself carries meaning. Alike things sit close together, and relationships become directions.

That leaves one question open. We said the computer learns where to place things. But what does it actually learn, and how? That's the next waypoint: Learn from examples.