AI dictionary · LLMs and GenAI
What is DPO?DPO kya hai?
Also called: Direct preference optimisation
DPO: A way to fine tune LLMs on pairs of preferred and rejected answers, simpler than RLHF.LLMs ko pasand aur na pasand jawaabon ke jodon pe fine tune karne ka tarika, RLHF se simple.
🧸 Think of it likeAise samjho
Like showing a trainee two replies and saying 'this one is better, that one is worse', again and again, until they naturally write the better kind.Jaise trainee ko do replies dikha kar baar baar kehna 'yeh behtar hai, woh kharaab', jab tak woh khud behtar wala likhna na seekh le.
📘 ExplainedVistaar se
Given a prompt with a better and a worse response, DPO adjusts the model to make the preferred response more likely, without training a separate reward model. It's widely used to align open models with human preferences after instruction tuning.Ek prompt ke behtar aur badtar response diye hon to DPO model ko adjust karta hai taaki pasand wala response zyaada likely ho, bina alag reward model train kiye. Instruction tuning ke baad open models ko insaani pasand se align karne mein kaafi use hota hai.
💡 Real exampleAsli example
Training a support model to prefer short, polite, cited answers over long vague ones.Support model ko lambe vague jawaabon ke bajaye chhote, vinamra, cited jawaab pasand karna sikhaana.
📖 Learn it step by stepStep by step seekho
🔗 Related termsJude hue shabd
Questions people ask
What is DPO?
A way to fine tune LLMs on pairs of preferred and rejected answers, simpler than RLHF. Given a prompt with a better and a worse response, DPO adjusts the model to make the preferred response more likely, without training a separate reward model. It's widely used to align open models with human preferences after instruction tuning.
How can I understand DPO in simple words?
Like showing a trainee two replies and saying 'this one is better, that one is worse', again and again, until they naturally write the better kind.
What is an example of DPO?
Training a support model to prefer short, polite, cited answers over long vague ones.
DPO ka matlab kya hai?
LLMs ko pasand aur na pasand jawaabon ke jodon pe fine tune karne ka tarika, RLHF se simple. Jaise trainee ko do replies dikha kar baar baar kehna 'yeh behtar hai, woh kharaab', jab tak woh khud behtar wala likhna na seekh le.
See DPO in the full A to Z dictionary · Start a free AI course