What is A/B test?A/B test kya hai?

🧸 Think of it likeAise samjho

Like a sweet shop selling the same laddoo in two box designs for a week, then keeping the box that sold more.Jaise mithai ki dukaan ek hafte tak same laddoo do alag box designs mein bechti hai, phir jo box zyada bika woh rakh leti hai.

📘 ExplainedVistaar se

Users are split at random into a control group (the current version) and a treatment group (the new version). Because the split is random, the only systematic difference between the groups is the change, so a difference in results can be called a real effect rather than a coincidence. You decide the main metric and the guardrail metrics before starting, run for the planned time, and check whether the difference is statistically significant.Users ko randomly control group (purana version) aur treatment group (naya version) mein baanta jaata hai. Split random hai, isliye groups ke beech systematic fark sirf badlaav ka hota hai, aur results ka fark sach mein asar maana ja sakta hai, sanyog nahi. Main metric aur guardrail metrics shuru se pehle tay karo, planned time tak chalao, aur check karo ki fark statistically significant hai ya nahi.

💡 Real exampleAsli example

Half of the users see AI reply suggestions, half don't; you compare how fast tickets get solved.Aadhe users ko AI reply suggestions dikhte hain, aadhon ko nahi; phir compare karte ho tickets kitni jaldi solve hue.

📖 Learn it step by stepStep by step seekho

• Metrics and A/B testing for AI featuresAI features ke liye metrics aur A/B testing
• Correlation vs causation, and sampling biasCorrelation vs causation, aur sampling bias

🔗 Related termsJude hue shabd

• Guardrail metricA metric that must not get worse while you improve the main metric.Aisa metric jo main metric sudhaarte waqt bigadna nahi chahiye.
• MetricA number you track to judge whether something is working.Ek number jo track karte ho yeh jaanne ke liye ki kuch kaam kar raha hai ya nahi.