AI dictionary · Evaluation and safety
What is LLM as judge?LLM as judge kya hai?
Also called: Model graded eval
LLM as judge: Using an LLM with a rubric to grade another model's answers.Rubric ke saath ek LLM se doosre model ke jawaabon ko grade karwana.
🧸 Think of it likeAise samjho
Like a senior teacher using a marking scheme to grade answer sheets written by another teacher's class.Jaise senior teacher marking scheme se doosri class ki answer sheets check kare.
📘 ExplainedVistaar se
It scales evaluation of open ended answers (helpfulness, faithfulness, tone). Judges have biases (prefer longer answers, the first option, their own style), so give a clear rubric, ask for reasoning, check agreement with human ratings, and use code checks when the answer can be verified exactly.Isse open ended jawaabon (helpfulness, faithfulness, tone) ka evaluation bade scale pe hota hai. Judges ke biases hain (lambe jawaab, pehla option, apna style pasand), isliye saaf rubric do, reasoning maango, human ratings se agreement check karo, aur jab jawaab exactly verify ho sake tab code checks use karo.
💡 Real exampleAsli example
A judge model scoring each RAG answer 1 to 5 for 'supported by the sources'.Judge model har RAG jawaab ko 'sources se supported' ke liye 1 se 5 score deta hai.
📖 Learn it step by stepStep by step seekho
🔗 Related termsJude hue shabd
Questions people ask
What is LLM as judge?
Using an LLM with a rubric to grade another model's answers. It scales evaluation of open ended answers (helpfulness, faithfulness, tone). Judges have biases (prefer longer answers, the first option, their own style), so give a clear rubric, ask for reasoning, check agreement with human ratings, and use code checks when the answer can be verified exactly.
How can I understand LLM as judge in simple words?
Like a senior teacher using a marking scheme to grade answer sheets written by another teacher's class.
What is an example of LLM as judge?
A judge model scoring each RAG answer 1 to 5 for 'supported by the sources'.
LLM as judge ka matlab kya hai?
Rubric ke saath ek LLM se doosre model ke jawaabon ko grade karwana. Jaise senior teacher marking scheme se doosri class ki answer sheets check kare.
See LLM as judge in the full A to Z dictionary · Start a free AI course