What is Autoscaling?Autoscaling kya hai?

🧸 Think of it likeAise samjho

Like a supermarket opening more billing counters when the queue gets long and closing them when it's quiet.Jaise supermarket line lambi hone pe zyada billing counters khol deta hai aur khaali hone pe band kar deta hai.

📘 ExplainedVistaar se

When load rises, more copies (replicas) of the service start; when it falls, they're removed to save cost. Kubernetes' Horizontal Pod Autoscaler scales on CPU, memory or custom metrics; for LLM servers, queue length or tokens per second are better signals. Models take time to load, so plan headroom and readiness checks.Load badhne pe service ki aur copies (replicas) start hoti hain; ghatne pe cost bachaane ke liye hata di jaati hain. Kubernetes ka Horizontal Pod Autoscaler CPU, memory ya custom metrics pe scale karta hai; LLM servers ke liye queue length ya tokens per second behtar signals hain. Models load hone mein time lete hain, isliye headroom aur readiness checks plan karo.

💡 Real exampleAsli example

During a festival sale the recommendation service grows from 4 to 20 replicas, then shrinks at night.Festival sale mein recommendation service 4 se 20 replicas ho jaati hai, phir raat ko ghat jaati hai.

📖 Learn it step by stepStep by step seekho

• Kubernetes: running containers at scaleKubernetes: containers ko bade scale pe chalaana
• Serving LLMs efficiently: GPUs, batching and costLLMs ko efficiently serve karna: GPUs, batching aur cost

🔗 Related termsJude hue shabd

• KubernetesA system that runs and manages many containers across many servers: starting, scaling and healing them.System jo bahut saare servers pe bahut saare containers chalata aur sambhaalta hai: start, scale aur theek karna.
• LatencyHow long a request takes from asking to getting the answer.Request bhejne se jawaab milne tak kitna time lagta hai.
• InferenceUsing a trained model to make predictions on new data.Trained model se naye data pe predictions karna.