What is Quantisation?Quantisation kya hai?

🧸 Think of it likeAise samjho

Like saving a photo in a smaller file size: it looks almost the same but takes far less space.Photo ko chhote file size mein save karne jaisa: dikhti lagbhag same hai par jagah bahut kam leti hai.

📘 ExplainedVistaar se

Instead of 16 bit numbers, weights are stored in 8 or 4 bits, cutting memory by 2 to 4 times with usually small quality loss. It lets bigger models fit on cheaper GPUs or even laptops. Always re-check quality on your eval set after quantising.16 bit numbers ke bajaye weights 8 ya 4 bits mein rakhe jaate hain, memory 2 se 4 guna kam, aam taur pe thode quality loss ke saath. Isse bade models saste GPUs ya laptops pe bhi fit hote hain. Quantise karne ke baad hamesha apne eval set pe quality dobara check karo.

💡 Real exampleAsli example

A 14 GB model becomes about 4 GB in 4 bit form.14 GB ka model 4 bit roop mein lagbhag 4 GB ka ho jaata hai.

📖 Learn it step by stepStep by step seekho

• Serving LLMs efficiently: GPUs, batching and costLLMs ko efficiently serve karna: GPUs, batching aur cost
• Fine-tuning and LoRA: specialising a modelFine-tuning aur LoRA: model ko specialise karna

🔗 Related termsJude hue shabd

• GPUA chip that does thousands of maths operations in parallel, which makes training and running AI fast.Chip jo hazaaron maths operations ek saath karta hai, jisse AI ki training aur chalaana tez hota hai.
• LoRAA cheap fine tuning method that trains small add-on matrices instead of the whole model.Sasta fine tuning tarika jo poore model ke bajaye chhote add-on matrices train karta hai.
• ParameterA number inside a model that is learned during training; LLMs have billions of them.Model ke andar ka number jo training mein seekha jaata hai; LLMs mein arabon hote hain.