Vector search is a retrieval method that represents queries and documents as vectors (ordered lists of numbers) and returns the documents whose vectors are closest to the query vector. Because the numbers are produced by a model trained to place similar meanings near each other, vector search can match a question to a passage even when the two share no words.
It is the mechanism behind most semantic search systems and one of the main ways AI search tools find passages to read before writing an answer.
How does vector search work?
A vector search system has an indexing phase, done in advance, and a query phase, done each time someone searches.
- Split the content. Documents are cut into chunks, usually a paragraph or a few hundred tokens long. Each chunk becomes one searchable item.
- Embed each chunk. An embedding model reads the chunk and outputs a vector, for example a list of several hundred or a few thousand numbers.
- Store the vectors. The vectors go into a vector index, together with a pointer back to the original text and any metadata such as the URL or date.
- Embed the query. When a question arrives, the same embedding model turns it into a vector of the same length.
- Find the nearest neighbors. The index returns the stored vectors closest to the query vector, and the system looks up the text they point to.
The query and the documents must be embedded with the same model. Vectors from two different models live in different spaces, and distances between them mean nothing.
What is an embedding?
An embedding is the vector a model produces for a piece of text. Each position in the list is a dimension, and no single dimension has a readable meaning on its own. The information is in the position of the whole vector relative to others: texts about similar things end up close together, and unrelated texts end up far apart.
Embedding models are neural networks, often related to the architecture of a large language model, trained on pairs of texts that should match, such as a question and its answer. Training pushes matching pairs closer and non-matching pairs apart. A model trained this way places "how do I reset my router" near a passage titled "Restoring factory settings on your modem".
Closeness is measured with a similarity function. The common ones are:
- Cosine similarity, which compares the angle between two vectors and ignores their length.
- Dot product, which multiplies matching dimensions and adds the results. For vectors normalized to the same length, it ranks results the same way as cosine similarity.
- Euclidean distance, which measures the straight-line distance between two points.
How does a vector index find matches quickly?
Comparing a query against every stored vector is exact but slow once an index holds millions or billions of chunks. Most production systems use approximate nearest neighbor (ANN) search instead, which finds vectors that are very likely among the closest without checking all of them.
Two families of ANN methods are widely used:
- Graph-based indexes, such as HNSW (hierarchical navigable small world), link each vector to some of its neighbors. A search starts at an entry point and walks the graph toward the query, moving to whichever neighbor is closer at each step.
- Cluster-based indexes, such as IVF (inverted file index), group vectors into clusters around center points. A search compares the query to the centers first, then scans only the vectors in the nearest few clusters.
Both give up a small amount of accuracy for a large gain in speed. Settings such as how many clusters to scan or how wide the graph search is let the operator choose where on that trade-off the system sits. Vectors can also be compressed to save memory, which lowers precision a little further.
How is vector search different from keyword search?
Keyword search, also called lexical search, matches the words in a query against the words in documents, usually with a scoring formula such as BM25 that rewards rare terms appearing often. Vector search matches on meaning, and each approach handles some cases better than the other.
| Keyword search | Vector search | |
|---|---|---|
| Matches on | Shared words | Similar meaning |
| Synonyms and paraphrases | Missed unless added manually | Usually found |
| Exact strings (product codes, names, error messages) | Found reliably | Sometimes missed or blurred |
| Index structure | Inverted index of terms | Vector index of embeddings |
| Explaining a result | Easy: the matching words are visible | Harder: closeness has no readable reason |
Because the weaknesses are complementary, many systems run both and merge the results. This is called hybrid search. A common merging method, reciprocal rank fusion, combines the positions of each result in the two lists instead of their raw scores, since keyword scores and similarity scores are on different scales.
Where does vector search fit in AI search?
AI search tools use retrieval to give a language model current text to answer from, a pattern called retrieval-augmented generation. Vector search is often one of the retrieval steps. A web search engine may supply candidate pages, which are then chunked and embedded so the system can pick the passages closest to the question.
The nearest neighbors are usually a shortlist. A reranker (a slower model that reads the query and each passage together) typically rescores them before the best few reach the language model, a later stage covered in how AI search chooses which sources to cite. Vector search decides which passages are in the running at all.
What does vector search mean for a website?
Vector search scores chunks, so the unit that competes is a passage. A page can be relevant overall and still lose if no single chunk of it is close to the question. A paragraph that states its topic, names the product or concept it is about and answers one question tends to produce an embedding that sits near queries on that topic.
Several practical consequences follow:
- Passages that rely on earlier context ("as mentioned above", "this feature") embed poorly, because the chunk loses the subject when it is cut out.
- Long sections that mix several topics produce vectors that sit between those topics and match none of them strongly.
- Exact names, model numbers and brand names are where vector search is weakest, so stating them in plain text also helps the keyword half of a hybrid system find the page.
- Headings that match how people phrase questions give the chunk under them a clear subject.
Generative engine optimization recommends the same writing habits, and the mechanism described here is a large part of the reason.
