Lesson · ⏱ 7 min read

🚪 Serving a model as an API with FastAPI

Q: Why load the model at startup instead of inside the predict function?

Loading on every request would make each request slow. Loading a model can take seconds. Do it once and reuse it for every request.

🏠 Real-life example

A restaurant kitchen (your model) shouldn't take orders from customers directly. A waiter (the API) takes a clear order from a menu, passes it to the kitchen, and brings back the dish. Other apps, like a website or mobile app, talk to your model only through this waiter.

💡 In simple words

A model in a notebook helps nobody. Wrapping it in a web API makes it usable by any app. FastAPI is the popular Python framework: you define an endpoint (like POST /predict), describe the input with a Pydantic model (which validates types automatically), load the model once at startup, and return a JSON response. FastAPI also generates interactive docs at /docs. Add a health-check endpoint, input validation, timeouts and logging. These are what make it production-ready.

🎬 See how it works

Apps never touch the model directly. They send a request to the API endpoint, the API runs the model, and a JSON response comes back.

🪜 Step by step

Load the model once when the app starts, not on every request.
Define the input with a Pydantic model so FastAPI validates types and required fields automatically.
Write a POST /predict endpoint that preprocesses, predicts and returns JSON with the prediction and model version.
Add GET /health for load balancers, and log each request's latency and errors.
Run it with uvicorn, test it with the automatic /docs page, and write a few automated tests.

🔑 Key terms

Endpoint: A URL plus method that does one job, e.g. POST /predict.
Pydantic model: A class describing the expected input, with automatic validation.
Latency: How long a request takes to get a response.

👀 See it in action

python

from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI()
model = joblib.load("price_model.pkl")   # loaded once at startup

class Flat(BaseModel):
    area_sqft: float
    bhk: int

@app.post("/predict")
def predict(flat: Flat):
    price = model.predict([[flat.area_sqft, flat.bhk]])[0]
    return {"price_lakh": round(price, 1)}

Run with: uvicorn app:app. Any app can now POST a flat and get a price.

⚠️ Common mistakes to avoid

No input validation: a missing field crashes the service or gives garbage predictions.
Slow work inside the request, like loading files or calling slow services without timeouts.

✅ Quick check

Why load the model at startup instead of inside the predict function?

It uses less disk
Loading on every request would make each request slow
It's required by Python

Show answer

B. Loading on every request would make each request slow: Loading a model can take seconds. Do it once and reuse it for every request.

🙋 Questions learners ask

Why FastAPI and not Flask?

Both work. FastAPI gives automatic validation, async support and interactive docs, which suit ML APIs well.

How do I handle many requests at once?

Run several worker processes or replicas behind a load balancer, and batch predictions where the model supports it.

📌 Remember

Model → API endpoint → any app can use it. Validate inputs and log everything.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

ML Engineer · AI Product Engineer · MLOps / AI Platform Engineer · LLM / Agent Engineer · AI Forward Deployed Engineer

Continue in the course