Lesson 17 — Beginner

LLM Capabilities & Failure Modes

Har model apni taakat ke saath chalta hai aur apni kamiyon ke saath bhi. Is lesson mein hum do taraf dekhenge — LLM kis tarah ke kaam naturally accha karta hai, aur kis tarah ke kaam par tootta hai. Aur sabse zaroori: hallucination hota kyun hai, aur engineering mein sawaal "can it do this?" ki jagah "how reliably" kaise poochhte hain.

⏱ 10 min• Beginner• The Context Window

LLM ki common strengths

WHAT

LLM ki strength hai unstructured text ko samajhna aur badalna. Summarization, transformation, extraction, classification, translation, coding, brainstorming — jahan text hai, wahan model aage nikalta hai.

WHEN

Jab input bhi text ho aur output bhi — jahan rules likhna mushkil ho (regex fail ho jaaye), jahan structure dene ki zarurat ho, jahan creativity chahiye.

WHERE

Support-ticket triage, email drafting, document summarization, code assistant, content transformation — AI features ka lagbhag har use-case isi par khada hai.

Text transformation: rewriting, tone change, grammar fix, format badalna.
Extraction aur classification: JSON mein facts nikaalo, tickets ko category mein daalo.
Semantic understanding: "is review mein complaint hai ya compliment" — keywords se aage ja kar arth samajhna.
Few-shot adaptation: 2-3 example de kar apna custom format seekh lena — bina code change ke.

Common failure modes

Hallucination: fluent galat jawab — sabse famous kami.
Arithmetic mistakes: exact calculation (paise, totals, dates) par gadbad aam.
Brittle reasoning: multi-step logic mein beech mein galat turn le leta hai.
Instruction-following failures: purane model ya mushkil prompt par rules tod deta hai.
Prompt sensitivity: prompt ki chhoti si wording se output bigad jaata hai.
Outdated knowledge: cutoff ke baad ka nahi pata — phir bhi confident guess.
Nondeterminism: dobara chalao toh alag wording, kabhi kabhi alag conclusion.

Yaad rakho: Kamiyon ka pattern yahi hai — model ko verification ki machine nahi banaya gaya, likhne ki machine banayi gayi hai. Isliye uske output ko verify karne wala layer chahiye — tools, checks, ya insaan.

Hallucination hoti kyun hai — mechanism

Hallucination ko mystery se nahi, mechanism se samjho. Model ka kaam hai likely continuation banana — sach verify karna uska kaam hi nahi. Jab uske paas sahi jankari nahi hoti, tab bhi wo probability ke hisaab se lagne wala likh deta hai — aur confident tone bhi wahi rehti hai. Yahi hallucination hai.

4 tarah ke sawaal banao, tone compare karo:

A. Model ko pata hai   → "India ki rajdhani?"             → sahi + confident
B. Model ko pata nahi  → "<naye startup ke 2026 facts>"   → guess, phir bhi confident
C. Misleading context  → document mein galat fact daalo   → usko repeat kar sakta hai
D. Info insufficient   → "Kal ka mausam kya hoga?"        → bina bole guess

Jab tak A aur B ka tone same hai, tab tak tone par bharosa mat karo.

Pro Tip: Hallucination kam karne ke tareeke — context mein sahi fact do (retrieval), jawab ko evidence se bandho (citations), aur na-pata sawaal par abstain karne ko bolo. Iske poore tareeke RAG (L4) aur Security (L8) levels mein hain.

Engineering sawaal — can it se how reliably tak

Engineering ka sawaal badlo: "Can GPT do this?" ki jagah poochho — "How reliably does this model perform this task under the conditions of my application?" Reliability kisi leaderboard se nahi, apne data par test karne se milti hai.

10 sawaal banao: kuch jinke jawaab model ko pata hai, kuch jinke nahi, kuch misleading context wale, kuch insufficient info wale. Inhi par model chalao aur dekho ki accuracy, tone aur refusal kaise badalte hain.
Production ke liye: apne real examples ka chhota eval set banao (poora tareeka L7 mein) aur har naye model ya version par wahi dohrao.
Expectation set karo: LLM ka output ek draft hai, final nahi. Human-in-the-loop wahi lagao jahan galti mehngi ho.

Exercise

Question: LLM ka sabse common failure mode — galat lekin fluent jawab — kya kehlaata hai?

Question: Hallucination ka mechanism kya hai — verify karne wali machine hai ya likely continuation banane wali? (do shabd, English mein)

Question: Engineering sawaal "Can it do this?" ki jagah kya poochho? (do shabd, English mein)

Question: Exact calculation (paise, totals) LLM pe chhod dena sahi hai — haan ya na?

Common mistakes

Confident tone ko proof samajhna: Model ka tone training se aaya hai, sach se nahi — galat jawab bhi utni hi confidence se aata hai. Tone par kabhi bharosa mat karo, evidence aur test results par karo.
Ek demo se production decision lena: Ek accha demo se reliability ka andaza nahi lagta. Apne 10 real examples par test karo — wahin asli farq dikhta hai, demo mein nahi.
Arithmetic aur dates ko LLM pe chhod dena: Exact totals, interest aur dates ke calculations tool ya code se karo — LLM sirf decide karega ki kya calculate karna hai. Galti yahan mehngi padti hai.

Desi example: Chacha ji ka confident funda — hallucination ka desi analogy

Har parivaar mein ek chacha ji hote hain jo har topic pe itminan se bata dete hain — source ki zarurat nahi. Kabhi sach, kabhi anjaan, par tone hamesha ek jaisi. LLM bhi aisa hi hai — fark sirf ye hai ki unke paas duniya bhar ka padha hua hai:

Tum: "Ye naya startup jaante ho?"
Chacha ji (confident): "Haan haan, uske founder mujhe jaante hain…"
LLM (confident): "<jo bhi probability se lagta hai>…"

Sach pata ho toh dono sahi; nahi pata toh dono guess — tone alag nahi.

Interview angle — yahan se sawaal aate hain

Q: LLM ki common strengths kya hain? — Summarization, transformation, extraction, classification, semantic understanding, coding, translation aur few-shot adaptation.
Q: Hallucination kyun hota hai? — Model likely continuation banata hai, verify nahi; jankari na ho toh bhi probability se likh deta hai, confident tone ke saath.
Q: "Can it do this?" mein kya galat hai? — Ye demo-level sawaal hai; asli sawaal "how reliably, mere conditions mein" — reliability apne examples par test se milti hai.
Q: LLM se exact calculation karwaana kyun khatarnak hai? — Numbers token-pieces mein tootte hain aur model verify nahi karta — isliye tool ya calculator use karo.

Lesson complete?

Capabilities aur unki kamiyan clear. Agla lesson — Understanding the Model Landscape — mein dekhenge closed API, open weights aur managed platforms teeno raste aur unke trade-offs.