What is Attention?Attention kya hai?

🧸 Think of it likeAise samjho

Like reading a sentence with a highlighter: to understand the word 'it', your eyes jump back to the words that tell you what 'it' is.Highlighter ke saath sentence padhne jaisa: 'it' samajhne ke liye aapki nazar wapas un shabdon pe jaati hai jo batate hain ki 'it' kya hai.

📘 ExplainedVistaar se

Each token creates a query, a key and a value vector. Comparing a token's query with every key gives attention weights (how relevant each other token is), and the token's new representation is a weighted mix of the values. So 'bank' near 'river' ends up different from 'bank' near 'loan'. Attention cost grows with the square of the text length, which is why long contexts are expensive.Har token ek query, ek key aur ek value vector banata hai. Token ki query ko har key se compare karne pe attention weights milte hain (baaki token kitne relevant hain), aur token ki nayi representation values ka weighted mix hoti hai. Isliye 'river' ke paas wala 'bank' 'loan' ke paas wale 'bank' se alag ban jaata hai. Attention ka kharch text ki lambai ke square ke saath badhta hai, isliye lambe contexts mehenge hain.

💡 Real exampleAsli example

In 'The animal didn't cross the road because it was tired', attention links 'it' to 'animal'.'The animal didn't cross the road because it was tired' mein attention 'it' ko 'animal' se jodta hai.

📖 Learn it step by stepStep by step seekho

• Transformers and attention: how LLMs readTransformers aur attention: LLMs kaise padhte hain

🔗 Related termsJude hue shabd

• TransformerThe neural network architecture behind modern LLMs, built around attention.Aaj ke LLMs ke peeche ka neural network architecture, attention pe bana.
• KV cacheMemory that stores past attention keys and values so an LLM doesn't recompute them for every new token.Memory jo purane attention keys aur values rakhti hai taaki LLM har naye token pe unhe dobara na nikaale.
• TokenA chunk of text, often part of a word, that LLMs read and write; pricing and limits are counted in tokens.Text ka tukda, aksar shabd ka hissa, jo LLMs padhte aur likhte hain; pricing aur limits tokens mein gine jaate hain.