Lesson 59 — Intermediate

Grounding, Citations & Abstention

RAG ka asli faayda tab milta hai jab har claim ka source traceable ho — aur kabhi kabhi sahi jawab yehi hai ki "mujhe nahi pata". Is lesson mein hum evidence se jawabna, claims ko sources se jodna, citations ≠ grounding ka farq, aur insufficient evidence pe abstain dekhenge.

Grounding kya hai

Retrieval tumhara kaam hai, generation LLM ka — par dono ke beech ka trust contract tum define karte ho. "Sirf context se bolo, cite karo, warna abstain karo" — yehi teen rules grounding banate hain.

Claims ko sources se jodna

def grounded_prompt(context_blocks, question):
    lines = []
    for i, b in enumerate(context_blocks, start=1):
        lines.append("[{}] {}\n{}".format(i, b["heading"], b["text"]))
    numbered = "\n\n".join(lines)
    return TEMPLATE.format(context=numbered, question=question)

# output contract:
# - har claim ke saath [n]
# - last line: "sources: [1], [3]"    <- UI mein dikhao

• Context blocks ko pehle se number karo — model sirf wahi numbers use karega.
• Answer parse karo: [n] nikaal ke original chunk id se map karo — UI mein clickable source.
• Har sentence ke saath citation compulsory banao — chhota answer, tight contract.
• Citation ko chunk ke saath store karo (doc_id + heading + page) — user ko dikha sakein kahan se aaya.

Citations ka matlab grounding nahi

import re
def cited_ids(answer):
    return {int(x) for x in re.findall(r"\[(\d+)\]", answer)}

def groundedness_check(answer, n_blocks):
    bad = [i for i in cited_ids(answer) if i > n_blocks]
    return len(bad) == 0     # bahar ka citation = fabricated source

# par sirf ye check kaafi nahi:
# [2] lagaya bhi ho, par us block mein claim ka support hai ya nahi —
# uske liye claim-source match chahiye (LLM judge ya keyword overlap)

• Format check: citation number existing block hai ya nahi — ye toh sirf syntax hai.
• Support check: cited block sach mein wahi baat bolta hai ya nahi — asli groundedness yahan hai.
• Model kabhi kabhi citation lagata hai par content us block se "borrowed" hota hai — ye fake grounding hai.
• Doosri taraf kabhi model sahi baat bolta hai bina citation ke — bina source ke trust nahi.

Kab abstain karna hai

NO_ANSWER = "Insufficient information — is corpus mein iska jawab nahi hai."

def answer_or_abstain(query, hits, min_score):
    if not hits or hits[0]["score"] < min_score:
        return {"text": NO_ANSWER, "citations": [], "abstained": True}
    prompt = GROUNDED.format(context=assemble(hits), question=query)
    ans = chat(prompt)
    # prompt mein bhi likha hai: context mein na ho toh "Insufficient information"
    return {"text": ans, "citations": cited_ids(ans), "abstained": False}

• Abstain ke do layer: retrieval (score threshold) aur generation (prompt rule) — dono saath kaam karte hain.
• Abstain = ek valid answer, error nahi — UI mein "try karo: ..." ya related docs ka option do.
• Log mein abstain rate track karo: bahut zyada = retrieval weak, bahut kam = model guess kar raha hoga.
• Forced answers sabse bade trust-killers hain — ek galat jawab 10 sahi jawab ki credibility kha jaata hai.

Exercise

Common mistakes

• Citation ko grounding samajhna: [2] lag gaya, par us block mein support hai ya nahi check hi nahi kiya. Fix: format check ke saath support check bhi karo — LLM judge ya overlap se rough verify.
• Bina contract ke citation maangna: Prompt mein number format na bataya toh model [4] jaisa invent karega jo exist hi nahi karta. Fix: numbered context + explicit output contract.
• Hamesha jawab dena: Score 0.1 pe bhi model ko force kiya — hallucination guaranteed. Fix: retrieval threshold + prompt abstain rule, dono.

claim  : "refund 7 din mein aata hai"
cite   : [3]  <- page exist karta hai?      FORMAT CHECK
support: page 3 pe "7 business days" likha?  SUPPORT CHECK

page nahi mili -> abstain; page hai par baat alag -> fake citation