Lesson 99 — Intermediate

Responsible AI & Governance

Technical controls ke upar ek aur layer hai: governance — kis use-case par model chalega, kya document hoga, kiski nazar rahegi, aur kya record bachega. Is lesson mein core concerns (hallucination, bias, oversight), model cards, EU AI Act aur India DPDP jaise frameworks ke practical implications, aur auditability banane ka tareeka.

⏱ 10 min• Intermediate• Red Teaming AI Systems

Governance kya hai

WHAT

Governance ka matlab: decisions likhe hue hain — kya use allowed hai, kya nahi; kaise evaluate kiya; kahan insaan ki nazar hai; aur kaun zimmedar hai. Ek document + process, ek lecture nahi.

WHEN

Jab AI kisi insaan ke against decision le ya unhe prabhavit kare — hiring, credit, medical, legal, HR, public services. Jahan stakes badhte hain, wahan governance mandatory hota hai.

WHERE

Model cards, risk assessments, evaluation records, approval workflows, audit logs aur review cadence mein — engineering artifacts ke saath chalta hai, unke alag nahi.

Funda: jo cheez document nahi hoti, woh defend bhi nahi ho sakti. Interview mein bhi aur production mein bhi — "tumhara system kis limit tak safe hai?" ka jawab sirf tab convincing hota jab likha ho.

Engineers ka role: Governance sirf legal/compliance ka kaam nahi — tum hi decide karte ho kya log karna hai, kya store hoga, task par model ko kitna leash. Yeh sab design decisions hain.

Core concerns

Hallucination: model confident galat bolta hai — citation/verification layer, aur high-stakes jawab par insaan.
Bias/fairness: training data aur task framing se skew aata hai — evaluation slices banao (har group par alag metrics), sirf average mat dekho.
Transparency: user ko pata chale ki AI hai, kis liye use ho raha hai, aur uska output kitna reliable — dark patterns mat banao.
Accountability: har automated action ka ek owner — "model ne kiya" koi jawab nahi hai.
Human oversight: override, escalation aur appeal ka raasta — high-stakes par human-in-the-loop default.

In sab ka ek practical test hai: Galat hone par kya hoga? Kiske against? Kaise pata chalega? Kaise theek hoga? Yeh chaar sawaal likh lo — concern khud actionable ban jaata hai.

Fairness ka sach: Perfect fairness ka koi single metric nahi hai. Jo bhi definition choose karo, use document karo aur trade-offs likho — chhupana hi sabse zyada risk hota hai.

Document karo — model card

model_card:
  intended_use: "Internal HR FAQ — employees ke policy sawaal"
  out_of_scope:
    - "termination ya disciplinary decisions"
    - "legal ya medical advice"
  known_limits:
    - "policy PDFs ke bagair hallucinate kar sakta hai — citations zaroori"
    - "regional languages mein quality alag-alag ho sakti hai"
  risks:
    - name: "Policy interpretation mein bias"
      mitigation: "HR sampling review har month"
  human_oversight: "low-confidence answers HR queue mein jaate hain"
  evals: "faithfulness evals + L8 adversarial suite"
  versions:
    model: "<asli model id>"
    system_prompt: "v3"
    last_reviewed: "<date>"

# Card chhota rakho (1-2 pages), lekin har model/prompt upgrade par update —
# stale card = no card.

Model card ka audience tumhara apna team hai bhi, aur risk review wale bhi. Ismein intended use, out-of-scope, known limits, risks + mitigations, oversight mechanism, evaluation results aur versions — yehi content har serious AI team rakhti hai.

Card ka faayda: Naya engineer onboarding par yahi padhta hai. Support ticket aaye toh "card mein out-of-scope tha" — yehi accountability ka proof hai.

Regulation aur auditability

EU AI Act (mechanism samjho): risk-based tiers — kuch uses unacceptable (banned), kuch high-risk (risk management, logging, human oversight, documentation jaise extra duties), kuch limited (transparency duties), kuch minimal. Jitna zyada risk, utni zyada obligations.
India ka DPDP framework (mechanism): personal data ke liye consent, purpose limitation, data fiduciary ki duties (notice, deletion, breach notification), aur children ke data par extra rules. Tumhara app agar India mein users ka data rakhta hai, toh yeh applicable hota hai.
Dono ka practical matlab: inventory rakho — kaunsa data, kis purpose par, kiske against decision, kaise delete hoga. Specific obligations ke liye apne legal counsel se verify karo; frameworks badalte hain.

# Audit trail — ek JSONL line per decision (append-only)
AUDIT = {
    "ts": "2026-10-19T10:22:03Z", "user": "u_1042",
    "action": "refund_lookup", "tool": "get_ticket",
    "args_hash": "9f2c...", "decision": "allowed",
    "model": "<model id>", "prompt_version": "v3", "approver": None,
}
AUDIT_2 = {
    "ts": "2026-10-19T10:24:41Z", "user": "u_1042",
    "action": "send_email", "tool": "send_email",
    "args_hash": "1b70...", "decision": "approved_by_human",
    "model": "<model id>", "prompt_version": "v3", "approver": "ops_lead_7",
}
# Rule: kaunne action liya, kis version (model+prompt) par, kisne approve —
# teeno jawab milne chahiye. Yehi auditability hai: logs sirf debugging
# nahi, accountability hain. PII mat rakho — hashes aur IDs kaafi hain.

Version pinning: Audit tab meaningful hai jab model id, prompt version aur policy version logged hon. "Pichhle hafte wala answer" trace karne ka doosra tareeka koi nahi.

Exercise

Question: Intended use, limits aur risks document karne wala card — do shabd, English?

Question: India ke DPDP framework mein data sambhalne wali entity ka naam, do shabd?

Question: EU AI Act ke risk tiers mein sabse strict tier jismein kuch uses band hain — naam? (do shabd)

Question: Action, version aur approver ke records rakhne ka principle — do shabd, English?

Common mistakes

Governance ko sirf legal team ka kaam samajhna: Log kya hai, task par model kitna autonomous hai, escalation kahan — yeh sab engineering decisions hain. Design time par decide karo, compliance review par nahi.
Model card ek baar bana kar bhool jaana: Har model/prompt upgrade par limits badalti hain. last_reviewed date rakho aur upgrade checklist mein card update mandatory karo.
High-stakes task se oversight hata dena: Credit, medical, HR, legal — yahan human-in-the-loop default hai. Bina oversight ke na accountability banti hai, na legal defence.

Desi example: Dawai ka label

Har dawai ke saath label hota hai: kis liye, kiske liye nahi, side effects, kisko doctor ki zaroorat. Bina label ki dawai pharmacy mein nahi bikti. Model card wahi label hai AI product ka:

dawai label   = intended_use + out_of_scope + side_effects
model card    = intended_use + out_of_scope + known_limits
# Label ke bina dawai unsafe; card ke bina high-stakes AI deploy mat karo

Interview angle — yahan se sawaal aate hain

Q: Responsible AI ke core concerns kaunse? (hallucination, bias, transparency, accountability, human oversight — har ek ka mitigation plan)
Q: Model card mein kya hoga? (intended use, out-of-scope, known limits, risks + mitigations, oversight, evals, versions + last reviewed)
Q: EU AI Act ka mechanism kya hai? (risk-based tiers — unacceptable se minimal; high-risk par logging, oversight, documentation jaise duties)
Q: Auditability kaise banoge? (append-only logs: action, args hash, decision, model+prompt version, approver — PII ke bina)

Lesson complete?

L8 ke saare lessons complete. Ab capstone — Red-Team & Secure an Agentic RAG App — mein sab jod ke ek poora secured app banate hain.