Lesson · ⏱ 6 min read

⚖️ AI literacy for decisions: build vs buy, cost and limits

Q: A company wants an internal Q&A bot over its policies. What's usually the right first approach?

A model API with RAG over the policy documents. API + RAG is cheap, fast and keeps answers current. Training from scratch is overkill.

🏠 Real-life example

For your shop's billing, you could hire developers to build software, or subscribe to a ready app for ₹500 a month. If billing isn't what makes your shop special, you buy. The same choice comes up with AI: use an existing AI product or API, customise one, or build your own model.

💡 In simple words

Most companies should not train their own foundation models. The usual ladder is: buy an off-the-shelf AI tool → use a model API with good prompts → add RAG over your data → fine-tune → build custom models. Climb only when the step below fails. Costs include API tokens, engineering time, evals, monitoring and the human review needed for errors. Know the limits: models hallucinate, can be inconsistent, struggle with exact calculation unless given tools, and data privacy rules may restrict which providers you can use. Good AI decisions weigh value, cost, risk and time to launch.

🪜 Step by step

Define the need precisely: task, volume, quality bar, latency and data sensitivity.
Check what exists: a SaaS product, a model API, an open model you can host.
Test the top two options on your own eval set, not on vendor demos.
Compare total cost over a year: usage, infrastructure, engineers to build and maintain, and switching cost.
Decide, and keep an exit path: wrap vendors behind your own interface so you can switch.

🔑 Key terms

Foundation model: A large general-purpose model (GPT, Claude, Llama) that others build on.
Total cost of ownership: All costs over time: API, people, monitoring and errors.
Vendor lock-in: Becoming dependent on one provider in a way that's hard to change.

👀 See it in action

text

Need: summarise 2,000 customer calls/day
Buy:   speech-analytics SaaS ≈ ₹2 lakh/month, live in 2 weeks
Build: transcription + LLM API ≈ ₹60k/month + 2 engineers × 3 months
Decision: buy now, revisit build if volume ×5 or custom insights are needed

A clear, numbers-based decision with a trigger for revisiting it.

⚠️ Common mistakes to avoid

Building custom models for problems a good API already solves well.
Comparing only per token price and ignoring engineering and maintenance time.

✅ Quick check

A company wants an internal Q&A bot over its policies. What's usually the right first approach?

Hire a research lab
Train a foundation model from scratch
A model API with RAG over the policy documents

Show answer

C. A model API with RAG over the policy documents: API + RAG is cheap, fast and keeps answers current. Training from scratch is overkill.

🙋 Questions learners ask

How do I estimate LLM cost per month?

Requests per month × (input tokens × input price + output tokens × output price). Add a margin for retries and growth.

Are open source models good enough?

For many tasks, yes, especially after fine tuning. Test them on your eval set against an API model before deciding.

📌 Remember

Buy → API → RAG → fine-tune → build. Climb only when needed.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

AI Product Manager

Continue in the course