What is Evals?Evals kya hai?

🧸 Think of it likeAise samjho

Like a fixed set of practice tests you rerun after every change, to see if the system got better or worse.Practice tests ka ek fixed set jo har badlaav ke baad dobara chalaate ho, yeh dekhne ke liye ki system behtar hua ya kharaab.

📘 ExplainedVistaar se

For LLM apps, define what good means, collect real examples (common, edge cases, past failures), and score them with code checks where possible and rubrics (human or LLM judge) where needed. Run evals on every prompt or model change, like unit tests, and add each new production failure. Evals are what separate demos from reliable products.LLM apps ke liye accha ka matlab tay karo, asli examples ikattha karo (common, edge cases, purane failures), aur jahan ho sake code checks se, jahan zaroori rubrics (insaan ya LLM judge) se score karo. Har prompt ya model change pe unit tests ki tarah evals chalao, aur har naya production failure jodo. Evals hi demos ko reliable products se alag karte hain.

💡 Real exampleAsli example

100 real customer questions with expected answers, rerun after every prompt change.Expected jawaabon ke saath 100 asli customer sawaal, har prompt change ke baad dobara chalaaye.

📖 Learn it step by stepStep by step seekho

• Evaluating AI: how to know it actually worksAI ko evaluate karna: kaise pata chale ki sach mein kaam karta hai
• Writing an AI product spec (PRD) with an eval planEval plan ke saath AI product spec (PRD) likhna

🔗 Related termsJude hue shabd

• LLM as judgeUsing an LLM with a rubric to grade another model's answers.Rubric ke saath ek LLM se doosre model ke jawaabon ko grade karwana.
• BenchmarkA standard test set used to compare models fairly.Models ko nishpaksh tarike se compare karne ke liye ek standard test set.
• HallucinationWhen an AI confidently states something false or made up.Jab AI confidently kuch galat ya gadha hua bolta hai.