Vector Embedding

What Is Vector Embedding?

A vector embedding is a list of numbers that represents the meaning of a piece of text, code, or an image, produced by a machine learning model. Content with similar meaning gets numbers that sit close together in that numeric space, which lets software find related items by measuring distance instead of matching keywords.

Key Takeaways

• Embeddings turn meaning into geometry. "Reset my password" and "I can't log in" land near each other even though they share no words.
• They are the retrieval layer behind semantic search and retrieval-augmented generation, which is how agents find relevant docs and code without stuffing everything into the context window.
• Embeddings from different models are not interchangeable. Switching embedding models means re-embedding your entire corpus, so budget for that migration before you pick one.
• Embedding calls are cheap compared to generation, usually a small fraction of the per-token price of a chat model, so the real costs are storage, indexing, and re-embedding when content changes.

How It Works

An embedding model reads an input, runs it through a neural network, and outputs a fixed-length vector, typically somewhere between 256 and 3,072 dimensions depending on the model. The model was trained so that inputs which appear in similar contexts produce similar vectors. The lineage runs back to Google's 2013 word2vec paper, whose architectures could learn high-quality word vectors from a 1.6-billion-word dataset in less than a day [1]. Similarity between two vectors is usually measured with cosine similarity: a score near 1 means the meanings are close, a score near 0 means they are unrelated.

To make this useful at scale, you embed every document, code file, or ticket once, store the vectors in a vector database or a plain index, and then embed each incoming query at request time. The database returns the nearest stored vectors, and those become candidate results. Most production systems chunk long documents before embedding, since one vector for a fifty-page PDF averages away the details a query actually cares about.

For engineers building with agents, the practical detail is that embeddings are how an agent's tools decide what to retrieve. When a coding agent searches a large codebase or a support agent pulls the right policy document, an embedding lookup usually did the narrowing. The quality of chunking and the choice of embedding model often matter more to output quality than the chat model on top. Model choice deserves a real evaluation rather than a default: the 2022 MTEB benchmark, which spans 8 embedding tasks across 58 datasets and 112 languages, found across 33 evaluated models that no single embedding method dominates every task [2].

Example

A team wires a coding agent into a monorepo with 40,000 files. Grep alone fails on questions like "where do we throttle outbound webhooks", because the code says rateLimiter and dispatchQueue, never "throttle". So they embed every function with docstrings into a vector index. Now the agent embeds the question, retrieves the twelve nearest functions, and reads only those files. Retrieval takes milliseconds, the agent's context stays small, and answers cite the actual throttling code on the first try.

What People Get Wrong

The common mistake is treating embedding similarity as a relevance judgment. Nearest-neighbor search returns whatever is closest, even when nothing in the corpus actually answers the query. A question about refund policy will still return twelve chunks if the corpus contains zero refund content, and a model downstream will happily ground its answer in them. Production systems need a similarity threshold, a reranking step, or both, so that "closest" is allowed to mean "not good enough".

FAQ

What is the difference between embeddings and tokens? Tokens are how a model reads text: small chunks of characters processed in sequence. Embeddings are a separate model's output: one numeric vector summarizing the meaning of a whole passage. Tokenization is an input step, embedding is a representation you store and compare.

Do embeddings work for code? Yes, and modern embedding models are trained on code specifically. They map a natural-language question and the function that answers it to nearby vectors, which is why semantic code search beats keyword search in large repos where naming is inconsistent.

How do I choose embedding dimensions? Larger vectors capture more nuance but cost more to store and search. Many current models support shortening vectors at query time, so a common approach is to start around 512 to 1,024 dimensions and only go bigger if retrieval quality measurably improves in your evals.

Sources

1. Mikolov et al., Google. "word2vec training speed on a 1.6-billion-word dataset." https://arxiv.org/abs/1301.3781. Accessed August 2026.
2. Muennighoff et al. "MTEB benchmark scope and finding that no embedding method dominates." https://arxiv.org/abs/2210.07316. Accessed August 2026.