An embedding is a list of numbers that represents the meaning of a piece of text, such as a word, a sentence or a passage. Texts with similar meaning get similar lists of numbers, so a computer can find related content by measuring the distance between them. Search engines, AI assistants and recommendation systems all use embeddings to match a question with the text that answers it.
The list is called a vector, and each number in it is a dimension. A typical embedding has hundreds or a few thousand dimensions. The meaning is spread across all of them, so you won't find one number with a readable label like "price" or "color", and a vector only becomes useful when it is compared with another.
How does an embedding represent meaning?
Picture a map where every sentence is a point. Sentences about the same topic sit close together, and unrelated sentences sit far apart. An embedding is the coordinate of one sentence on that map, except the map has hundreds of directions instead of two.
Because position encodes meaning, two texts can be close even when they share no words. "How do I cancel my subscription?" and "Steps to end a monthly plan" land near each other. "Apple released a new phone" and "I baked an apple pie" land far apart, even though both contain the word apple, which is something keyword matching cannot do.
How is an embedding created?
An embedding model produces the vector. It is a neural network, often built on the same design as a large language model, trained so that texts with related meaning end up with nearby vectors. During training it sees many pairs of texts that belong together, such as a question and its answer, and learns to pull them closer while pushing unrelated pairs apart.
Turning a web page into embeddings usually follows these steps:
- Split the page into chunks. Long documents are cut into passages of a few sentences or paragraphs, because one vector for a whole page would blur many topics together.
- Break each chunk into tokens, the small text pieces a model reads.
- Run the tokens through the embedding model, which outputs one fixed-length vector per chunk.
- Store each vector with a pointer back to the original text, usually in a database built for vectors.
When someone searches, their question goes through the same model. The query and the stored passages must be embedded by one model, because vectors from different models live on different maps and cannot be compared.
How are embeddings compared?
Systems compare two vectors with a similarity measure. The most common is cosine similarity, which looks at the angle between two vectors: a small angle means similar meaning. Some systems use the dot product or the straight-line distance. Each of these reduces two long lists of numbers to a single score.
Comparing a query with millions of stored vectors one by one would be slow. Vector search uses indexes that find the nearest vectors approximately, giving up a small amount of accuracy for a large gain in speed.
Where do AI search systems use embeddings?
Embeddings appear at several points between a question and a written answer.
- In retrieval, semantic search embeds the query and returns passages whose vectors sit closest, which finds relevant text written in different words.
- Hybrid search merges vector results with exact keyword matches, so a product code or a brand name is not lost when the meaning-based match is loose.
- In retrieval-augmented generation, embeddings help pick the passages a language model reads before it writes, grounding its answer.
- After a page is fetched, its passages can be scored against the question and the closest ones kept. How AI search chooses which sources to cite describes this ranking stage.
- Near-identical vectors reveal pages that say the same thing, so a system can group them and avoid citing several copies of one text.
What are the limits of embeddings?
Embeddings capture general meaning well and fine detail less well. The weaknesses follow from how the vectors are built.
| Weakness | What happens |
|---|---|
| Exact terms | Rare names, part numbers and codes may not be represented precisely, so a near match can outrank the exact one. |
| Negation | "Works with Mac" and "does not work with Mac" can produce very similar vectors. |
| Numbers | Prices, dates and quantities carry little weight in the vector compared with the topic. |
| Long chunks | A passage that covers several subjects gets an averaged vector that matches none of them strongly. |
| Model changes | Switching to a new embedding model requires embedding every stored passage again. |
Because of these gaps, many systems pair vector retrieval with keyword search and add a reranking step that reads the query and passage together more carefully.
What do embeddings mean for a website?
An AI search system usually judges a page in chunks, and each chunk gets its own vector. A section that sticks to one question and answers it in the opening sentences produces a vector close to that question. If a section mixes setup, pricing and history in one block, its vector sits between several topics.
Wording matters as well. A passage that uses the plain terms people type into a search box lands nearer to their queries than one built on internal jargon. Exact facts such as model numbers and prices are best stated in text where keyword matching can also find them, since the embedding alone may not separate them from similar values. Writing clear headings and keeping to one topic per section helps, and so do self-contained paragraphs, because each gives a chunk a sharper position on the map that retrieval reads from.
