What is Cross entropy?Cross entropy kya hai?

🧸 Think of it likeAise samjho

Like a quiz that gives a small minus for an unsure wrong answer but a huge minus when you were 100% sure and still wrong.Ek quiz jaisa jo unsure galat jawaab pe thoda minus deta hai, par 100% sure hokar galat hone pe bahut bada minus.

📘 ExplainedVistaar se

Cross entropy compares predicted probabilities with the true class. Predicting 0.9 for the right class gives a small loss; predicting 0.01 for it gives a large loss. LLMs are trained with cross entropy on the next token. Combined with softmax, its gradient is simply prediction minus truth.Cross entropy predicted probabilities ko sahi class se compare karta hai. Sahi class ko 0.9 dena chhota loss deta hai; use 0.01 dena bada loss. LLMs agle token pe cross entropy se train hote hain. Softmax ke saath iska gradient bas prediction minus sach hai.

💡 Real exampleAsli example

A cat image predicted as 95% dog gives a big cross entropy loss.Billi ki image ko 95% kutta predict karna bada cross entropy loss deta hai.

📖 Learn it step by stepStep by step seekho

• Gradients and gradient descent: how models improveGradients aur gradient descent: models behtar kaise hote hain
• Neural networks: layers that learnNeural networks: layers jo seekhti hain

🔗 Related termsJude hue shabd

• Loss functionA formula that measures how wrong the model's predictions are; training tries to make it small.Formula jo naapta hai model ki predictions kitni galat hain; training ise chhota karne ki koshish karti hai.
• SoftmaxA function that turns a list of scores into probabilities that add up to 1.Function jo scores ki list ko aisi probabilities mein badalta hai jinka jod 1 ho.