Lesson 175 — Advanced

Project & portfolio defense

Project defense ka matlab ye nahi ki project kaam karta hai — wo toh demo mein bhi dikhta hai. Asli test ye hai ki har technology choice ke peeche ka decision tum bata sakte ho, har number ke peeche ka measurement, aur har galti ke peeche ka lesson. Is lesson mein tum 2-3 projects ko ek structure mein pack karoge (problem → requirements → baseline → architecture → decisions → trade-offs → failures → evals → results), choices ko numbers se defend karoge, failures ko khule aam karoge, aur un follow-ups ka jawab pehle se taiyaar rakhoge jo har interviewer poochhta hai.

2-3 projects chuno — depth, gallery nahi

Chunne ka rule: JD se match + depth + measurable outcome. Ek project RAG wala, ek agent wala, ek deployment/LLMOps wala — teeno category cover ho toh kisi bhi sawaal ka anchor mil jaata hai. Gallery (10 chhote projects) se behtar hai 3 gehre.

Project story ka structure — ek line se shuru, numbers pe khatam

project_one_pager:            # har project ke liye pehle se likho
  problem:        kis user ki kis problem ko solve kiya (ek line)
  requirements:   scale, latency, quality, cost, privacy (kya non-negotiable tha)
  baseline:       purana tareeka aur uska number (ya "koi nahi tha" + proxy)
  architecture:   boxes + arrows (4-6 boxes se zyada nahi)
  decisions:      3-4 major choices + unka alternative jo reject kiya
  tradeoffs:      har decision ki cost (kya sacrifice kiya)
  failures:       kya toota, kaise pata chala, kya badla
  evals:          kaise naapa (golden set, offline + online)
  results:        numbers: quality, latency, cost/query, throughput

• Problem se shuru karo: user, pain, aur success criteria. "Maine LLM lagaya" problem nahi hai, solution ka hissa hai.
• Baseline compulsory hai: improvement ka paimana baseline se hi banta hai. Bina baseline ke "80% accurate" ka koi matlab nahi.
• Architecture chhota rakho: 4-6 boxes, arrows ke saath. Detail interviewer ke sawaal se aayegi, presentation se nahi.
• Numbers real hon: apne measurements — quality score, p95 latency, cost per query, dataset size, traffic. Estimate ho toh "estimate hai" bolo.

Har choice defend karo — model se deployment tak

choice -> alternative -> reason (har major decision ke liye):
model X       | model Y      | apne eval set pe X ka accuracy better, cost budget ke andar
rerank stage  | no rerank    | top-k recall badha — numbers dikhaye before/after
vector+keyword| vector only  | exact identifiers (policy numbers) embeddings se chhoot rahe the
framework A   | framework B  | team ko debugging aani chahiye; abstraction chhupa raha tha
DB Q          | DB P         | hybrid filter + ACL pre-filter chahiye tha, dono native
self-hosted   | API          | privacy constraint; traffic steady tha, break-even defend kiya

• Model selection: benchmark nahi, apna eval set — aur "budget ke andar" wali line real honi chahiye.
• Retriever aur chunking: kyun ye strategy? Numbers dikhao: hit-rate ya recall before/after.
• Agent architecture: workflow kyun nahi? Raasta ka unpredictable hona hi reason hai — trajectory se defend karo.
• Framework: framework ka naam kam, debugging/ownership ka point zyada. "Team debug kar sake" ek asli engineering reason hai.
• Database aur deployment: filters, ACL, latency, ops burden — enterprise interviewer yahin atak jaata hai, aur yahin score hota hai.

Failures khule aam — wahi sabse strong move hai

Interviewer ko pata hai ki har project mein kuchh toota tha. Jo candidate apni galti, uska detection aur uska fix teeno bata deta hai, uski credibility sabse oonchi ho jaati hai. Failures chhupana risky hai — interviewer ko lagta hai ki ya toh experience nahi hai, ya imandari nahi.

• Format: kya toota → kaise pata chala (metric/eval/user) → kya badla. Teen line, khul kar.
• Design mistakes: pehle chunking galat thi, ACL embedding mein chhupi thi, agent ko zyada tools de diye the — sab chalega, bas lesson saath ho.
• Process mistakes: eval set baad mein banaya, baseline skip kiya, do changes ek saath ship kiye — ye bhi engineering maturity dikhate hain.
• Lesson ko link karo: "isiliye ab main har deploy pe eval gate lagata hoon" — failure ko current practice se jodo, story ko regret se nahi.

Follow-ups jo har interviewer poochhta hai

mock Q&A outline (apne project pe likho):
Q: "Why not RAG?"        -> knowledge ki zarurat thi ya behaviour ki? fresh facts = RAG
                           case nahi tha toh seedha bolo; RAG ka frame bhi bata do
Q: "Why not fine-tune?"  -> volume/eval set nahi the; prompt+few-shot se baseline mil raha tha
Q: "Why an agent?"       -> raasta predictable nahi tha; trajectory dikhao jo decide kiya
Q: "100x traffic?"       -> serving layer pehle tootega: queues, rate limits, GPU capacity;
                           phir cache/staleness, ingestion lag — order ke saath
Q: "Halve the cost?"     -> levers: result cache, model routing (chhota model pe easy tasks),
                           prompt/context trimming, batch processing non-urgent kaam
Q: "What breaks first?"  -> apne system ka sach: ek bottleneck, uska metric, uska fix plan

• "Why not RAG/fine-tune/agent?": teeno ka frame tumhare paas ready ho (L15.4 aur L15.3) — decision ko product brief se defend karo, fashion se nahi.
• "100x traffic": order se bolo — serving layer (queues, rate limits, capacity), phir caches aur staleness, phir ingestion. Ek saath sab bolna = structure nahi hai.
• "Halve the cost": levers ki list ready rakho — result cache, model routing, prompt trimming, batching. Aur "pehle cost kahan ja raha hai wahi naapunga" wali line rakho.
• "What breaks first": ye sawaal system ki honesty test hai. Sachcha bottleneck bolo, uska metric batao, fix ka plan batao — "kuchh nahi tootega" se behtar hai.

Exercise

Common mistakes

• Project ko feature list ki tarah bolna: "Maine X use kiya, Y use kiya" — decisions kahin nahi. Fix: problem → requirements → decisions → numbers wala structure, har project ke liye one-pager likho.
• Failures chhupana: Interviewer ko lagta hai ya experience nahi hai ya imandari nahi. Fix: kya toota → kaise pata chala → kya badla — teen line, khul kar bolo.
• Numbers ke bina daave: "Quality improve hui" — kaise naapa? Fix: quality score, p95 latency, cost per query, baseline comparison — jo measure kiya wahi bolo, estimate ko estimate bolo.
• Har project bol dena: 10 projects ka tour — kisi pe gehrai nahi. Fix: JD se 2-3 chuno, unhi ki poori taiyari, baaki ke naam sirf ek line mein.

problem       -> "tenant ko documents dhoondne mein ghante lagte the"
baseline      -> keyword search: 60% relevant top-3 (apna measurement)
decision      -> hybrid retrieval: exact IDs keyword se, meaning embeddings se
failure       -> ACL embedding mein chhupi thi; leak eval se pakdi, filter mein fix
result        -> relevant top-3: baseline se behtar (apna eval set, same examples)