Lesson 55 — Intermediate

RAG Architecture & When to Use It

Ab tumhare paas search engine hai — ab usse LLM se jodna hai. Is lesson mein hum RAG ka core pipeline, kis problem pe RAG chahiye aur kis pe nahi, aur retrieval failure aur generation failure ka farq dekhenge — ye diagnose karne ka skill poore baad ke level mein kaam aayega.

⏱ 10 min• Intermediate• Capstone — Build a Hybrid Search Engine

Core RAG pipeline

Kya hai?

RAG (Retrieval-Augmented Generation): user ke sawaal ko pehle relevant documents dhoondne ke liye use karna, phir un documents ko context ke roop mein LLM ko dekar grounded jawab banwana. Retrieval tumhara hai, generation LLM ki.

Kab kaam aata hai?

Private knowledge (company policies), aksar badalta data (pricing, inventory), bade knowledge bases jinhe model ne nahi dekha, aur jahan har jawab ka source traceable hona chahiye.

Kahan milta hai?

Support bots, internal knowledge assistants, legal/medical doc Q&A, aur kisi bhi product feature mein jahan "docs se poochho" likha ho. Tumhara capstone engine isi ka retrieval layer hai.

query -> rewrite (optional)     # M4.11
  -> retrieve (dense/sparse)   # M4.4 - M4.6
  -> rerank (optional)         # M4.10
  -> assemble context + ids    # M4.9
  -> generate (grounded prompt) # M4.9
  -> cite / abstain            # M4.12

Ek line mein: RAG = tumhara search engine (retrieval) + LLM (synthesis) + prompt (glue). Isliye RAG achha tabhi banega jab retrieval achha ho — is level ka pehla aadha hissa isi liye retrieval tha.

RAG kya solve karta hai

Private knowledge: model ne tumhare internal docs nahi dekhe — retrieval unhe har request pe laata hai.
Frequently changing data: prices, policies, inventory — fine-tune karte rahoge? Retrieval mein sirf index update.
Bade knowledge bases: laakhon docs — sab context mein nahi ho sakte, top-k chhunna hi toh kaam tha.
Source-grounded answers: "policy ke section 4 ke hisaab se..." — citation compliance aur audit ke liye zaroori.

RAG ka asli faayda updateability aur traceability hai — knowledge ko model ke andar nahi, system ke paas rakha hai. Isliye data badla toh index update, aur jawab ke saath source bhi mil jaata hai.

Kab RAG ki zarurat NAHI hai

Model already jaanta hai: general facts, common coding questions — RAG sirf latency aur cost add karega.
Pure transformation: "is text ko summarize karo" — input text tum already prompt mein de rahe ho, retrieval ki zarurat nahi.
Sab kuch context mein fit ho jaaye: chhota document set (kuch pages) — seedha prompt mein daal do, retrieval ka drama mat karo.
Ek specific skill chahiye: formatting, tone, domain jargon — kabhi kabhi fine-tuning ya prompting kaafi hai; RAG nahi.

Decision rule: Sawaal poochho: "Jawab ke liye kya mere paas chhote docs hain jo badalte rehte hain ya private hain?" Haan → RAG. Nahi → pehle prompt/fine-tune socho. RAG default nahi, ek tool hai.

Retrieval failure vs generation failure

def debug_answer(query, hits, answer):
    print("QUERY:", query)
    for h in hits:
        print(" HIT", h["id"], round(h["score"], 3), h["text"][:80])
    print("ANSWER:", answer)
    # hit galat/weak  -> retrieval failure
    #   fix: chunking, embedding, query mismatch, k, filters
    # hit sahi, jawab galat -> generation failure
    #   fix: prompt, context ordering, model, temperature

Retrieval failure: sahi chunk hi nahi mila. Jawab kuch bhi ho, model kuch bhi guess kare — root cause retrieval mein hai.
Generation failure: context mein sahi chunk tha, phir bhi jawab galat/udhaaru — prompt ya model ko dosh.
Pehle context print karo, bina answer dekhe. Aadhe sawaal isi step se solve ho jaate hain.
RAG ka 80% debugging yahi do categories mein aata hai — isliye har answer ke saath retrieved hits log karo.

Judge separately: Retrieval ko retrieval ke metric se judge karo (kya sahi chunk mila), generation ko answer quality se. Ek doosre ko blame karne se fix kabhi nahi milta — production mein yehi sabse common trap hai.

Exercise

Question: RAG ke core pipeline mein query ke baad sabse pehla step kya hai?

Question: Relevant chunk hi nahi mila — ye kaunsa failure hai?

Question: RAG ka full form kya hai?

Question: Jab zarurat ki knowledge chhoti ho jo context mein poori fit ho jaaye — RAG ki zarurat ya seedha prompt?

Common mistakes

Har problem pe RAG laga dena: General knowledge ya transformation tasks pe RAG sirf latency/cost laata hai. Fix: pehle decision rule — private/chhota-but-changing/traceable knowledge hai hi ya nahi.
Retrieval failure ko prompt se theek karne ki koshish: Sahi chunk hai hi nahi, prompt kitna bhi strong ho jawab guess hoga. Fix: pehle context dump dekho, phir chunking/retrieval pe jao.
Generation failure ko embedding pe blame: Chunk sahi tha par jawab galat — embedding kyun badlega? Fix: prompt aur context ordering check karo, judge separately.

Desi example: Reference desk ka librarian

Librarian se sawaal karo. Agar usne galat shelf se book la di (retrieval failure) toh jawab galat — chahe wo kitna bhi achha sun ke bata de. Aur sahi book laake andar ka page galat dikhaya (generation failure) — dono ka fix alag hai:

sawaal -> librarian dhoondhta hai (retrieve)
      -> khol ke padhta hai  (generate)

shelf galat  = retrieval failure  -> index/shelf system fix
page galat   = generation failure -> reading/prompt fix

Interview angle — yahan se sawaal aate hain

Q: RAG ka pipeline? — Query → (rewrite) → retrieve → (rerank) → assemble → generate → (cite/abstain).
Q: RAG kab nahi chahiye? — Model already jaanta ho, pure transformation, ya sab kuch context mein fit ho.
Q: Retrieval aur generation failure farq? — Pehle mein sahi chunk nahi mila, doosre mein chunk sahi tha par jawab galat; debug alag.
Q: RAG aur fine-tuning ka farq? — Knowledge external rakhna + update-friendly (RAG) vs model ke weights mein behavior sikhaana (fine-tune).

Lesson complete?

Architecture clear — ab tum decide kar sakte ho kab RAG banayein aur kis type ki failure dekh kar kahan fix karein. Agla lesson — Baseline RAG from Scratch — mein poora pipeline khud likhenge.