Lesson 173 — Advanced

AI system-design interviews

System design round ka asli test knowledge nahi hai — tum ambiguity ko kaise assumptions mein badalte ho, aur phir har decision ko trade-off ke saath defend karte ho. Is lesson mein tum ek design loop seekhoge jo har AI sawaal pe chalega: assumptions → simplest viable architecture → full AI stack ka incremental use → enterprise constraints (jahan public API hi allowed nahi) → aur chaar systems jinke pehle se whiteboard drafts bana rakhe hona chahiye.

⏱ 20 min• Advanced• LLMOps & production debugging interviews

Design loop — pehle sabse simple, phir complexity

Kya hai?

AI system-design interview mein tum ek product requirement ko ek chalti hui system mein badalte ho — model selection, RAG, tools, storage, observability, evals, security, deployment tak. Interviewer tumhari soch dekh raha hai, diagram ki finishing nahi.

Kab kaam aata hai?

Har senior-level AI loop mein, aur aksar take-home ke bajaye live whiteboard pe. "Design karo..." wala sawaal aate hi tumhara pehla kadam yehi hona chahiye.

Kahan milta hai?

Enterprise support assistant, coding assistant, research agent, large-scale document intelligence — chaaro classic hain, aur inke variations har jagah.

Loop ek hi hai, chaaron jagah: assumptions lo → sabse simple viable architecture banao → har added complexity ko ek problem se justify karo → bottlenecks, failure modes aur scaling batao. RAG, agents, fine-tuning, routing, caching — ye sab tabhi aate hain jab koi requirement unhe maange. Pehle din se sab kuch jod dena sabse aam galti hai.

Rule of thumb: Har added component ke liye ek line chahiye: "ye kis problem ko solve kar raha hai, aur iski cost kya hai". Bina is line ke wo impress karne wala keyword hai, architecture nahi — aur interviewer use turant pakadta hai.

Ambiguity → assumptions — 8 axis pe pehle se likho

assumption sheet (design shuru karne se pehle, 2 minute):
users         -> kaun? employees (SSO, ACL) ya public (anon, abusability)
traffic       -> roughly scale (queries/day) + burst pattern
latency       -> interactive (< few seconds) ya async (minutes chalega)
quality       -> galti ka cost (support reply vs medical content)
cost          -> budget ka rough frame (tokens per query ki ceiling)
freshness     -> knowledge kitni frequently badalti hai
privacy       -> data kahan ja sakta hai (PII, residency, compliance)
reliability   -> SLA expectation (99 ke aas-paas ya best effort)

jo assume kiya wo BOLDO — chhupana baad mein "maine ye maana tha" banta hai

Users aur traffic: public product pe abuse, rate limiting aur cost caps pehle se sochne hote hain; internal tool pe SSO aur ACL.
Latency aur quality: interactive assistant mein streaming chahiye, batch pipeline mein nahi. Galti ka cost high hai toh human-in-the-loop ya citations mandatory.
Freshness: roz badalta data = RAG + incremental ingestion; stable data = ek baar index, mahino chale.
Privacy aur reliability: residency constraint ya SLA — yehi do enterprise designs ko public API se alag kar dete hain.

Assumptions bolna hi skill hai: Interviewer jaan-boojh kar vague dega. "Do sawaal poochh loon assumptions clear karne ke liye?" — ye line professional deta hai. Aur phir assumptions whiteboard pe likh do, taaki baad mein "ye toh maine assume kiya tha" visible rahe.

Full AI stack — har layer ka kaam aur trade-off

box diagram (har layer apne box mein, arrows ke saath):

[user] -> [API gateway: auth, rate limit]
       -> [orchestrator: plan -> retrieve -> generate -> verify]
            [model selection/routing]  task ke hisaab se model (ya provider) chunta hai
            [RAG + context]            retrieval, rerank, context assembly, citations
            [tools/agents]             kewal jab raasta dynamic ho (L15.3 ka frame)
       -> [caches]                     semantic cache (same query) + embedding cache
       -> [databases]                  vector + search + relational (metadata, ACL)
       -> [inference services]         API ya self-hosted, batching, fallbacks
       -> [observability + evals]      har stage metrics + rolling quality score
       -> [security]                   IAM, secrets, encryption, injection guards
       -> [deployment]                 CI/CD, eval gates, canary, rollback plan

Model selection/routing: task ke hisaab se model — simple query pe chhota sasta model, mushkil pe bada. Routing ek cost/quality dial hai, decoration nahi.
Caches: LLM cache mein exact match se zyada kaam karta hai — semantic cache (query ka meaning same ho toh cached answer). Par staleness ka trade-off bhi bolo.
Databases: vector (embeddings), search index (keyword/hybrid), relational/graph (metadata, ACL, relations) — teeno ka alag kaam, ek DB mein sab nahi.
Observability aur evals: design mein hi rakho, baad mein nahi. "Quality naapne ka plan kya hai?" — ye sawaal aayega; answer pehle se likha ho.
Security aur deployment: auth, rate limits, injection guards; phir CI/CD with eval gates, canary rollout, rollback plan.

Trade-offs, tool names nahi: "Vector DB mein Qdrant/Weaviate/Pinecone..." se impress nahi hota. Bolo: "vector + keyword hybrid, kyunki exact identifiers (policy numbers) embeddings se chhoot jaate hain" — ye trade-off hai jo har DB pe laagu hota hai.

Enterprise constraints — jab public API hi allowed nahi

constraint: "data public API pe nahi ja sakta"

option A: managed platform (private deployment / VPC offering)
  + ops burden kam, features ready
  - region choice limited, cost premium, vendor surface bana rehta hai

option B: self-hosted open-weight
  + data poora andar, control poora
  - serving, scaling, patching, safety tuning sab tumhare
defend kis pe:
  identity    -> SSO/RBAC platform pe available ya khud banana padega?
  residency   -> region options constraint satisfy karte hain?
  capacity    -> peak traffic pe GPU quota available?
  cost        -> steady-state utilization se compare karo (per-token bill se nahi)

Constraint hi design ko chunta hai: public API allowed nahi toh choices do ho jaati hain — managed private deployment ya self-hosted. Dono ke dard bolo, phir apne numbers se defend karo.
Identity: enterprise mein SSO aur RBAC non-negotiable. Managed platform pe toh native, self-hosted pe tumhe khud integrate karna padega — engineering time bhi cost hai.
Residency aur capacity: region list dekho, GPU quota check karo. "Design hai par capacity nahi" = design incomplete hai.
Cost comparison honest rakho: managed ka premium vs self-hosted ka ops engineer ka time — dono ko ek table mein laao, kisi ko strawman mat banao.

Enterprise round ka pattern: Interviewer constraint daalega (residency, audit, quota). Usko "problem" mat samjho — constraint ko assumption sheet mein add karo aur do options with trade-offs wapas do. Yahi senior behaviour hai.

Chaar systems jinke drafts pehle se ready hon

enterprise support assistant (ek ka mini outline):
  users: employees, internal knowledge (policies, tickets)
  simple: hybrid retrieval (vector + keyword) + rerank + grounded answer + citations
  add when needed: ticket creation tool (HITL), semantic cache (repeat queries)
  ops: per-doc ACL, rolling eval on ticket outcomes, cost per resolved ticket
  break first: stale policy docs (freshness SLA) + burst after product launch

baaki teen bhi isi frame mein: requirements -> simple -> justified adds -> ops -> break first

Enterprise support assistant: ACL + citations pehle, tools baad mein — galti ka cost high hai (wrong policy answer).
Coding assistant: context engineering (repo map, relevant files) hi asli challenge hai; latency budget tight, streaming zaroori.
Research agent: agent ka genuine case — raasta unknown, par budget caps aur source tracking mandatory.
Large-scale document intelligence: ingestion hi hero hai — parse quality, chunking, batch pipeline, per-doc processing cost.
Har system ka "break first" point yaad rakho — 100x traffic, freshness, cost, ya quality. Ye sawaal aksar aakhri mein aata hai aur score karta hai.

Practise ka tareeka: Har system pe 20-minute timer laga kar aloud whiteboard karo — assumptions, boxes, trade-offs, break-first. Ek design 3 baar dohrao: pehla messy, doosra structured, teesra apne hi follow-ups ke saath. Yehi capstone mock ka bhi format hai.

Exercise

Question: Ambiguous design question ka pehla kadam kya hai — assumptions ya architecture?

Question: Design ki shuruaat sabse complex stack se hoti hai ya simplest viable architecture se?

Question: Data residency non-negotiable hai, public API allowed nahi — deployment option kaunsa defend karoge?

Question: Har sawaal ko ek hi model ki bajaye task ke hisaab se alag model pe bhejna kya kehlata hai? (do shabd)

Question: Design mein har major decision ke saath kya dete hain — tool ka naam ya trade-off?

Common mistakes

Pehle din se complex stack banana: RAG + agents + fine-tune + multi-region — bina kisi requirement ke. Interviewer ko koi justify nahi dikhta. Fix: simplest viable, phir har add ek problem se.
Tool names se impress karana: Qdrant, LangGraph, KServe bolne se architecture nahi banta. Fix: har tool ke peeche trade-off bolo — kaunsi problem, kaunsi cost.
Assumptions chhupana: Andhere mein design, baad mein farq. Fix: 8-axis sheet (users, traffic, latency, quality, cost, freshness, privacy, reliability) pehle whiteboard pe likho.
Observability aur evals ko baad mein chhod dena: "Quality kaise naapenge?" pe blank — design incomplete. Fix: metrics, eval set aur rollback plan design mein hi add karo.

Desi example: Ghar banana — pehle naksha, phir room

Design interview ghar banane jaisa hai: pehle assumptions (kitne log, kitna budget, kitni zameen), phir sabse chhota chalne wala ghar (ek room), aur har nayi deewar ek zarurat se (family badhi toh kamra). Bina naksha ke cement mangwana = bina assumptions ke RAG laga dena:

naksha (assumptions) -> kitne log (traffic), budget (cost), zameen (privacy/residency)
ek room (simplest)   -> retrieval + LLM + answer
har deewar justified -> family badhi (tools) -> rozana kaam badha (caches)
bijli-paani (ops)    -> observability, evals, rollback plan
kya pehle tootega    -> monsoon side (freshness) ya budget (cost)

Interview angle — yahan se sawaal aate hain

Q: Design shuru kaise karoge? — 8-axis assumptions bolo aur likho, phir simplest viable architecture; complexity sirf justified additions se.
Q: "Enterprise mein public API allowed nahi" — kya karoge? — Managed private deployment vs self-hosted, identity/residency/capacity/cost ke axes pe defend; dono ke dard bolo.
Q: Cache kahan use karoge aur kya risk? — Semantic cache repeat queries pe; staleness ka trade-off bolo aur TTL/invalidations ka plan.
Q: Model routing kyun? — Cost/quality dial: simple tasks pe chhota model, mushkil pe bada; routing ka metric (escalation rate) bhi naapo.
Q: Ye system 100x pe kya tootega? — Pehle serving layer (queues, rate limits, GPU capacity), phir cache/staleness aur ingestion lag — order ke saath bolo.

Lesson complete?

Design frame ready — ab tum har "design karo" sawaal ko assumptions se lekar break-first point tak le ja sakte ho. Agla lesson — Staying Current Without Chasing Hype — mein seekhenge ki ye framework naye models aur tools pe kaise apply karte ho bina fashion ke.