Room 4 · The Awakening · 2013

word2vec: Meaning as Geometry

word2vec: Meaning as Geometry (Tomas Mikolov and team · Google, 2013): Words as points in a space.

word2vec represents each word as a vector of numbers learned from large amounts of text. Words with similar meaning end up close together, and relationships become directions: the vector for “king” minus “man” plus “woman” lands very close to “queen”.

In 2013 Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean at Google published "Efficient Estimation of Word Representations in Vector Space," followed later that year by a second paper with more refinements. Together they introduced word2vec, a fast way to turn words into lists of numbers that capture something about their meaning.

The idea rests on an old principle from linguistics: you can know a word by the company it keeps. word2vec reads a large amount of text and trains a small network on a simple task, predicting a word from the words around it, or the surrounding words from a given word. The task itself is not what matters. What matters is the by-product: each word ends up represented by a vector, a list of a few hundred numbers, that summarizes the contexts in which it appears.

The resulting space has surprising structure. Words used in similar ways land near each other: "cat" near "dog," "Paris" near "Rome." And some relationships become consistent directions. The famous example is king minus man plus woman, which lands very close to queen. Similar arithmetic connected countries with their capitals and verbs with their tenses.

Google released the code and vectors trained on a large corpus of news text, and word2vec quickly became a standard tool in research and industry.

It also revealed a problem. Because the vectors learn from human writing, they absorb its biases. In 2016 researchers showed that the same arithmetic produced stereotyped associations between gender and professions, which opened an important line of work on fairness in language technology.

Today's language models still begin by turning each piece of text into a vector, a direct descendant of this idea.

Ask your AI: which words would be close to "museum" in a word2vec space, and why?

Did you know? It learned relationships like country and capital without anyone explaining them.

Source: arxiv.org/abs/1301.3781

See it in the museum →

← The Network That Found Cats · Generative Adversarial Networks →

All of Room 4: The Awakening

Cite this: Nazareno, I. (2026). word2vec: Meaning as Geometry (2013). New Mind Museum. Retrieved November 17, 2026, from https://newmindmuseum.com/museum/word2vec-meaning-as-geometry-2013/