AI Path Finder
Free consultation

Lesson · ⏱ 8 min read

🧲 Embeddings and vector databases: searching by meaning

EnglishHinglish

🏠 Real-life example

In a library, books on similar topics sit near each other. Cricket books are next to sports biographies, far from cookbooks. Embeddings give every piece of text a 'location' (a vector) so that similar meanings land close together. Then 'How do I return a damaged item?' sits right next to 'Refund policy for broken products', even with no shared words.

💡 In simple words

An embedding model turns text, or images, into a vector of hundreds of numbers that captures meaning. Similar meanings give vectors close together, measured with cosine similarity. A vector database (pgvector, Pinecone, Qdrant and others) stores millions of these vectors and quickly finds the nearest ones to a query. This is semantic search: searching by meaning, not exact keywords. It powers RAG, recommendations and duplicate detection. Combining it with keyword search (hybrid search) often works best for names and codes.

🎬 See how it works

UPI payment failedCard declinedTrack my deliveryDelivery is lateRefund policy for broken productsRaise a replacement requestQ: “How do I return a damaged item?”nearest = most similar ✓

Each text becomes a point. Similar meanings land close together, so the question finds the refund article even though they share no keywords.

🪜 Step by step

  1. Send text to an embedding model; you get back a vector of hundreds of numbers.
  2. Do this for every document chunk and store the vectors, with the text and metadata, in a vector database.
  3. At search time, embed the user's query the same way.
  4. Find the nearest vectors with cosine similarity; the vector database does this quickly even for millions.
  5. Return the matching chunks, optionally filtered by metadata (department, date, permissions).

🔑 Key terms

Embedding
A vector of numbers representing the meaning of text.
Cosine similarity
How close two vectors point. Near 1 means very similar.
Vector database
A database built to find the nearest vectors fast.

👀 See it in action

text
Query: "How do I return a damaged item?"
Similarity to stored help articles:
  0.89  Refund policy for broken or damaged products   ← top match
  0.74  How to raise a replacement request
  0.21  Track your delivery partner live

No keyword overlap with the top match, but the meaning matches. That's semantic search.

⚠️ Common mistakes to avoid

  • Mixing embedding models. Vectors from different models aren't comparable; re-embed everything when you switch.
  • Relying only on vector search for exact codes, names or IDs. Add keyword search (hybrid).

✅ Quick check

Why can semantic search find 'refund for broken product' for the query 'return damaged item'?

  1. Random luck
  2. They share keywords
  3. Their embeddings are close because their meanings are similar
Show answer

C. Their embeddings are close because their meanings are similar: Embeddings capture meaning, so similar ideas sit close together even with different words.

🙋 Questions learners ask

Do I need a special vector database?

Not always. For small projects, pgvector in PostgreSQL or even an in memory library works. Dedicated databases help at large scale.

Do embeddings work for Hindi?

Multilingual embedding models do, and can even match a Hindi question to an English document. Test with your own examples.

📌 Remember

Embeddings = meaning as numbers. A vector DB finds the nearest meanings fast.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

Data Engineer (AI focus) · AI Product Engineer · LLM / Agent Engineer · AI Forward Deployed Engineer

Continue in the course