Embeddings and Similarity
A vector is a list of floats; cosine similarity is six lines of Python, and near means related in use, not equal in meaning.
score = math.sumprod(a, b) / (math.hypot(*a) * math.hypot(*b))An embedding is a fixed-length list of floats produced from a piece of text, typically a few hundred to a few thousand numbers. The useful property is geometric: texts used in similar contexts land near each other. An embeddings endpoint takes strings and returns one vector per string; from Python's side the result is just a list[float].
Cosine similarity
Similarity is the cosine of the angle between two vectors: the dot product divided by the product of their lengths. That is sum(x * y for x, y in zip(a, b)) over math.hypot(*a) * math.hypot(*b). math.sumprod(a, b) (Python 3.12+) is the shorthand for that same dot product, computed in C and without the intermediate tuples. The result runs from -1 to 1, and for text embeddings in practice it sits between about 0 and 1.
What near means
Nothing absolute. A score of 0.82 is not a percentage and there is no universal threshold; useful cutoffs differ per model and per corpus, so find yours by looking at ranked results. What you can trust is the ranking: a higher score means more related than a lower one for the same query.
What trips people up
Embeddings capture topical similarity, not truth, logic or negation. "The build passed" and "the build failed" are close neighbours, because they are about the same thing. Use them to find candidate text, then let the model or your own code decide what the text actually says.
Example
When to use it
- Finding the support articles closest to a new question
- Collapsing reworded duplicates in a ticket queue
- Clustering survey answers by topic without predefined labels
- Ranking candidate chunks before assembling a retrieval prompt
More examples
Normalise once, then similarity is a dot product
The standard optimisation: embeddings never change after they are computed, so normalise them on the way in. Then each query does one multiply-and-add pass per document instead of two norms as well. Guard the zero vector, because an empty input string can produce one and the division would raise.
import math
def unit(vec: list[float]) -> list[float]:
norm = math.hypot(*vec)
if norm == 0.0: # an all-zero vector has no direction
raise ValueError("cannot normalise a zero vector")
return [x / norm for x in vec]
# Store unit vectors, and cosine similarity collapses to one sumprod: no
# square roots at query time, which is the whole cost on a large store.
store = [(doc.id, unit(doc.embedding)) for doc in docs]
def search(query_vec, k=5):
q = unit(query_vec)
scored = ((math.sumprod(q, v), doc_id) for doc_id, v in store)
return sorted(scored, reverse=True)[:k]Deduplicating near-identical text
A use for embeddings that is not search: collapsing reworded duplicates in support tickets, product reviews or scraped pages, where exact-match and hashing both fail. Assumes normalised vectors so the dot product is the cosine. The threshold is empirical, which is the honest answer for every embedding cutoff.
def dedupe(items, threshold=0.95):
"""Keep the first of each cluster of near-identical entries."""
kept = []
for text, vec in items:
if any(math.sumprod(vec, other) >= threshold for _, other in kept):
continue # a near-duplicate of something kept
kept.append((text, vec))
return kept
# 0.95 is not a universal number. Print the score distribution for your own
# corpus and pick the cutoff where real duplicates stop and topics begin.Where the vectors come from
Embedding in batches rather than one call per string, which is the difference between a minute and an hour on a large corpus. The two rules worth enforcing in code: results come back in request order and that is your only join key, and the model id is stored with the vector so you can tell when a store needs rebuilding.
# The Python side of an embeddings call, whichever provider supplies it:
# send a batch of strings, get one list of floats back per string, in order.
def embed_all(texts: list[str], batch=64) -> list[list[float]]:
vectors: list[list[float]] = []
for start in range(0, len(texts), batch):
chunk = texts[start:start + batch]
vectors.extend(embed_batch(chunk)) # one HTTP call per batch
assert len(vectors) == len(texts) # order is the only join key
return vectors
# Record the model id next to every vector. Scores across two different
# embedding models are not comparable, so a model change means a re-embed.
rows = [(t, v, "embed-model-id") for t, v in zip(texts, embed_all(texts))]
Discussion