Lesson · ⏱ 7 min read

🗂️ MLOps: experiment tracking and model registries

Q: A new model version performs badly in production. What lets you recover fastest?

Rolling back to the previous version in the model registry. The registry keeps tested versions, so rollback is a one-step change.

🏠 Real-life example

A good cook trying new biryani recipes writes down every attempt: rice brand, spice amounts, cooking time and how it tasted. Without notes, they can never recreate the best one. Experiment tracking is that notebook for ML, and a model registry is the shelf of approved, labelled recipes the restaurant actually serves.

💡 In simple words

MLOps is DevOps for machine learning: making model training reproducible and deployment reliable. Experiment tracking tools (MLflow, Weights & Biases) automatically log each run's code version, data version, parameters, metrics and output model, so you can compare runs and reproduce the best. A model registry stores approved model versions with stages like 'staging' and 'production', so deployments always pick a known, tested version and you can roll back instantly. CI/CD for ML adds automated tests and evals before a model is promoted.

🪜 Step by step

Log every training run: parameters, metrics, code commit and data version (MLflow or a similar tool).
Save the trained model and its preprocessing as one artifact.
Register good models in a model registry with a version and a stage (staging, production).
Promote a model only if it beats the current one on a fixed evaluation set.
Deploy by version, so rolling back is just pointing to the previous version.

🔑 Key terms

Experiment run: One training attempt with its settings and results logged.
Model registry: A versioned store of approved models and their stages.
Reproducibility: Being able to recreate the exact same model again.

👀 See it in action

python

import mlflow
with mlflow.start_run(run_name="xgb-depth4"):
    mlflow.log_params({"max_depth": 4, "lr": 0.05, "data": "orders_2026_09"})
    model.fit(X_train, y_train)
    mlflow.log_metric("recall", 0.81)
    mlflow.sklearn.log_model(model, "model", registered_model_name="fraud")

Every run is recorded, and the best one goes to the registry.

⚠️ Common mistakes to avoid

Keeping results in notebooks and spreadsheets. Nobody can reproduce or compare them later.
Changing the evaluation data between runs, so scores aren't comparable.

✅ Quick check

A new model version performs badly in production. What lets you recover fastest?

Retraining from memory
Deleting the logs
Rolling back to the previous version in the model registry

Show answer

C. Rolling back to the previous version in the model registry: The registry keeps tested versions, so rollback is a one-step change.

🙋 Questions learners ask

Is MLflow the only option?

No. Weights & Biases, cloud tools (SageMaker, Vertex AI) and others do similar things. The habit matters more than the tool.

How do I version data?

Use dated snapshots, table versions in your warehouse or lakehouse, or tools like DVC, and record the version with each run.

📌 Remember

Log every run, register every shipped model, and always be able to roll back.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

ML Engineer · MLOps / AI Platform Engineer

Continue in the course