Lesson 109 — Advanced

AI System Design Case Studies

Ab teen real systems whiteboard karte hain — document QA (fresh knowledge + citations), coding agent (retrieval + sandbox + long tasks) aur research agent (planning + parallel tools + budgets). Har ek ka process: requirements → baseline → architecture → bottlenecks → failure modes → scaling → trade-offs.

⏱ 20 min• Advanced• Monitoring, Feedback & Continuous Improvement

Method — boxes nahi, decisions

WHAT

2-3 systems whiteboard karna aur har box ka defence dena: kya requirement thi, kaunsa alternative reject hua, kya tootega. Boxes draw karna aasan hai — "kyun" answer karna asli skill hai.

WHEN

Interview system design round, architecture review, ya naye teammate ko design samjhate waqt. Teen classic patterns: doc QA, coding agent, research agent.

WHERE

Process har baar same: requirements → baseline → architecture → bottlenecks → failure modes → scaling → trade-offs. Har stage ka ek line ka note rakho.

Ek line ka rule: har box ke saath uska "kyun" aur uska rejected alternative. Interviewer ko diagram nahi, decision process chahiye — aur tumhara yahi difference dikhata hai.

Answer format: Requirement → constraint → option → choice → how measured. Ye paanch step har sawaal ka framework hai — ratta nahi, reasoning. Numbers guess karne ki zaroorat hi nahi padegi.

Case 1 — Document QA with fresh knowledge

Requirement: private docs se jawab, fresh (roz update), har jawab ka source dikhe — aur galat jawab ka asar high hai.
Knowledge flow: docs → parse/chunk/embed → ACL-tagged index + re-index schedule (L9.2).
Citations: answer par source id + page mandatory; citation na aaye toh response ko incomplete maano (validate karo).
Routing: simple lookup chhota model, synthesis wale sawaal bade (L9.3); repeated FAQ par cache (L9.5).
Escalation: confidence low/ACL miss/contradiction → human expert queue (L9.6 ladder).
Latency & cost: streaming start karo retrieval ke turant baad; top-k hi bhejo, poora corpus nahi.

user q -> guard -> cache? -> route -> retrieve (top-k + rerank)
       -> generate with citations -> validate: source attached?
       -> yes: stream answer
       -> no / low confidence: human queue
index  : daily docs -> parse -> chunk -> embed -> ACL-tagged index

Defend line: "Citation validate hoti hai, isliye user verify kar sakta hai; ACL retrieval par hi lagti hai, isliye leak hi nahi hota." — requirement se seedha decision.

Case 2 — Coding agent over a large codebase

Requirement: bade repo mein bug fix/features — context repo se bada, code chalana hai, tasks minutes-tak chalte hain.
Retrieval: symbol/graph search + file chunks; repo snapshot versioned taaki retrieval config traceable rahe (L9.8).
Tool & execution: shell/tests run → isolated sandbox (no prod creds, network policy, timeout).
Model selection: edit suggestions chhota fast model, architectural change bada model (L9.3).
Long-running: build/test steps async job + checkpoints, streaming status (L9.7).
Recovery: partial patch revert, test logs slim karke wapas context (L6), task budget guard.

Sandbox non-negotiable: Model-generated code kabhi bhi seedha host par mat chalao. Sandbox = filesystem/network/creds se cut environment, timeout ke saath — iske bina ye system design nahi, gamble hai.

Case 3 — Deep research agent

Requirement: ghante chalne wala research run — plan, kai sources, har claim ka source, budget ke andar.
Planning: goal ko chhote todo steps mein todo; har step ka apna budget.
Parallel tool calls: independent searches ek saath (L9.4); results slim (L6.6).
Context management: notes + checkpoints — long run mein window chhoti rakho (L6).
Citations: har claim source ke saath, source quality rank karo.
Budgets & recovery: token ceiling → exceed par partial report (explicit) + run state persist (L9.7).

plan = decompose(goal)                       # steps + budget per step
for step in parallel_ready(plan):
    results += gather(search(q) for q in step.queries)   # independent -> parallel
    notes.write(step, slim(results))           # checkpoint har step (L6.4)
    if budget.exceeded:
        return partial_report(notes)           # explicit partial > crash
report = synthesize(notes, citations=True)

Budget hi design hai: Research agent ka sabse bada failure cost/timeout blast hai. Per-step budget + partial report = system hamesha kuch na kuch deta hai, kabhi khali haath nahi.

Poora chain defend karo

Requirements → baseline: success criteria kya hai aur sabse simple version kya hoga.
Architecture → bottlenecks: scale par pehla kaun tootega — index? queue? model quota?
Failure modes: har box ke liye "ye fail hua toh" (L9.6 matrix).
Scaling: 10x traffic par kya badlega — sharding, queue, cache, batching (apne load test se).
Trade-offs: kya chhoda gaya aur kyun — "sab best hai" kabhi mat bolo.

Senior signal: Measurement maangna hi senior hai: "iska hit rate main measure karunga", "latency budget yahan split hoga". Numbers mat udao — method batao, numbers tumhare system denge.

Exercise

Question: Jawab ke saath source dikhane ka feature — ek shabd, English?

Question: Model-generated code chalane ka safe isolated environment — do shabd, English?

Question: Independent searches ko ek saath chalana — ek shabd, English?

Question: Har box ka "ye fail hua toh" wala section — do shabd, English?

Common mistakes

Boxes without reasons: Diagram ban gaya par har box ka "kyun" nahi. Har box ke saath requirement link + rejected alternative likho — warna defence nahi ho paayegi.
Fabricated numbers: "10ms hoga, 99% accurate" bina measurement ke — credibility khatam. Method batao: kya load test, kya eval set, kahan se number aayega.
Ek pattern har system par: Doc QA, coding aur research ke constraints alag hain (citations vs sandbox vs budgets). Pattern fit karo requirement par, copy-paste nahi.

Desi example: Wedding planner — case study ka desi analogy

Planner sirf list nahi banata — har decision ke paas requirement card hota hai (budget, guest count, venue). AI case study mein bhi har box ke paas wahi card hai:

requirement: "invite 300"   -> hall choose (why: capacity)
requirement: "answers + sources" -> RAG + citation validate (why: trust)
requirement: "runs 1 hour"   -> notes + budget guard (why: window + cost)

Interview angle — yahan se sawaal aate hain

Q: Document QA system kaise whiteboard karoge? (ingest → ACL index → route → top-k → generate + citation validate → escalate)
Q: Coding agent ke special concerns? (sandboxing, symbol retrieval, model selection, long-running checkpoints, recovery)
Q: Research agent ka budget kaise sambhaaloge? (per-step budget, checkpoints, partial report, citations har claim par)
Q: "Ye architecture 10x par tootega?" ka jawab kaise doge? (bottleneck pehle identify, phir sharding/queue/cache path + measurement plan)

Lesson complete?

Teen case studies defend ready. Final lesson — Capstone: poora L9 ek interview-ready design document mein likhna.