What is LoRA?LoRA kya hai?

🧸 Think of it likeAise samjho

Like adding a small clip on lens to your phone camera instead of buying a whole new camera.Naya camera kharidne ki jagah phone camera pe ek chhota clip on lens lagaane jaisa.

📘 ExplainedVistaar se

The base weights are frozen; LoRA learns a small low rank update for some layers, often under 1% of parameters. This cuts GPU memory and storage, and you can swap adapters per task. QLoRA keeps the base model in 4 bit precision so large models can be tuned on a single GPU.Base weights freeze rehte hain; LoRA kuch layers ke liye chhota low rank update seekhta hai, aksar 1% se kam parameters. Isse GPU memory aur storage kam lagti hai, aur har task ke liye adapters badal sakte ho. QLoRA base model ko 4 bit precision mein rakhta hai taaki bade models ek GPU pe tune ho sakein.

💡 Real exampleAsli example

A 30 MB LoRA adapter that makes a 7B model write in your company's report format.30 MB ka LoRA adapter jo 7B model ko company ke report format mein likhwata hai.

📖 Learn it step by stepStep by step seekho

• Fine-tuning and LoRA: specialising a modelFine-tuning aur LoRA: model ko specialise karna

🔗 Related termsJude hue shabd

• Fine tuningFurther training a pretrained model on your own examples to specialise its behaviour.Pretrained model ko apne examples pe aage train karke uska behaviour specialise karna.
• QuantisationStoring model weights with fewer bits so models use less memory and run faster.Model weights ko kam bits mein rakhna taaki model kam memory le aur tez chale.
• ParameterA number inside a model that is learned during training; LLMs have billions of them.Model ke andar ka number jo training mein seekha jaata hai; LLMs mein arabon hote hain.