Lesson · ⏱ 7 min read

🎛️ Fine-tuning and LoRA: specialising a model

Q: You want a model to answer from your company's latest policy documents, which change weekly. What should you try first?

RAG: retrieve the documents at question time. Fine-tuning teaches style and skills. For fresh, changing facts, RAG is cheaper and stays up to date.

🏠 Real-life example

An MBBS doctor already knows general medicine. To become a cardiologist they don't redo school. They spend a few focused years on hearts. Fine-tuning is the same: take a model that already knows language and train it a little more on your specific task. LoRA is like learning cardiology from compact notes instead of rewriting every textbook.

💡 In simple words

Pre-training a large model from scratch costs millions. Fine-tuning starts from a pre-trained open model (Llama, Mistral, Qwen) and continues training on your examples, such as support tickets with ideal replies. Full fine-tuning updates every weight, which is expensive. LoRA and other PEFT (parameter-efficient fine-tuning) methods freeze the original weights and train small add-on matrices, so you can fine-tune on a single GPU. QLoRA also compresses (quantises) the base model. Always compare against prompting and RAG first, because they're cheaper and often enough.

🪜 Step by step

First try prompting and RAG. Fine tune only if you need a consistent style, format or behaviour they can't give.
Collect a few hundred to a few thousand high quality examples of input and ideal output.
Choose a base model and use LoRA, so you train small adapter weights instead of the whole model.
Train for a few epochs, watching validation loss to avoid overfitting.
Evaluate against the base model on your eval set before using it. Keep the adapter versioned.

🔑 Key terms

Pre-trained model: A model already trained on huge general data.
LoRA / PEFT: Training small add-on weights instead of the whole model.
Quantisation: Storing weights with fewer bits to save memory.

👀 See it in action

python

from peft import LoraConfig, get_peft_model
config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(base_model, config)
model.print_trainable_parameters()
# trainable: 0.1% of all parameters → fits on one GPU

Only a tiny fraction of the weights are trained.

⚠️ Common mistakes to avoid

Fine tuning to add facts. It's unreliable for knowledge; RAG is better for that.
Low quality or inconsistent training examples. The model copies your mistakes faithfully.

✅ Quick check

You want a model to answer from your company's latest policy documents, which change weekly. What should you try first?

Fine-tune every week
RAG: retrieve the documents at question time
Pre-train a new model

Show answer

B. RAG: retrieve the documents at question time: Fine-tuning teaches style and skills. For fresh, changing facts, RAG is cheaper and stays up to date.

🙋 Questions learners ask

How much does fine tuning cost?

With LoRA on a small open model, it can run on a single GPU for a few hours. Hosted fine tuning APIs charge per training token.

What is QLoRA?

LoRA on a base model stored in 4 bit precision, which cuts memory so large models can be fine tuned on modest GPUs.

📌 Remember

Prompting → RAG → fine-tuning, in that order of cost.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

AI Research Engineer

Continue in the course