What's an embedding in AI, and what's it actually for?

A few months back I built the internal search for an online store that sells fishing gear, a client whose whole site we already look after. Someone typed “spinning rod” into the search box and got nothing back, even though the product was listed as “fishing rod for spinning” and was sitting right there in stock, photos and all. Old-school search compares words: if the query doesn’t roughly match the title or description letter for letter, you get zero results. I swapped the search engine for one built on embeddings and the problem was gone by the end of the afternoon, not after a week of hand-feeding it synonyms one by one.

An embedding is a list of numbers, a vector, usually with several hundred or a few thousand positions, that captures what a piece of text means rather than the letters it’s made of. “Spinning rod” and “fishing rod for spinning” end up as vectors that sit very close together even though they barely share any words, and that numerical closeness is what lets you search by meaning instead of by literal match.

To generate them I used OpenAI’s text-embedding-3-small: you send it text, it hands back a vector of 1,536 numbers, at around $0.02 per million tokens, so running the whole catalogue (about 4,000 products) through it cost pennies. I stored the vectors in Postgres using the pgvector extension, five lines of SQL for the index, and from there I could compare each search query’s vector against the products’ vectors and keep the closest ones. No new infrastructure, no bolting on an external service just for this.

What caught me off guard was how badly this explains itself to a client. I said “the system searches by meaning” and got a look like I was selling snake oil. What actually landed was screenshots, before and after: same search term, zero results with the old search, three relevant products with the new one. That took ten seconds to sink in; the theory wouldn’t have landed in ten minutes.

Embeddings are also the piece that lets a chatbot answer using a company’s own documents without the model ever being trained on them: you turn every chunk of a manual or a PDF into its own vector, and when someone asks a question, you pull the chunks whose vectors sit closest to that question and hand those to the model as context, as much as fits inside its context window. That’s the technical foundation of what most people call RAG, even though the term almost never comes up when I’m actually talking to a client about it.

What I do weigh carefully is which embedding model to use, because it’s not a one-size-fits-all choice: some are cheaper and faster, others cost more and pick up on finer distinctions, like unusual synonyms or the jargon of one specific industry. For the fishing catalogue the small model was more than enough; for a legal search tool with very specific terminology it probably wouldn’t be. I still don’t have a fixed rule for deciding that upfront. For now I just test both against a real sample of the client’s own searches before settling on one.