Lesson 107 — Advanced

AI Pipeline Orchestration

Ek AI feature mein kai components hain — guardrails, cache, routing, retrieval, model, tools. Orchestration yahi decide karti hai kaunsa step deterministic hai, kaunsa model par chhoda jaayega, aur kaam sync, async, queue ya long-running chalega.

⏱ 10 min• Advanced• Reliability, Guardrails & Graceful Degradation

End-to-end flow

WHAT

Orchestration = har request ka rasta: kaunse steps honge, kaunsa fixed hai, kaun model par jaayega, aur state kahan save hogi. Components ko jod kar ek coherent system banana.

WHEN

Jab ek se zyada components hain — cache, guardrails, retrieval, routing, tools. Single prompt wale app mein bhi flow sochna padta hai, bas wo chhota hota hai.

WHERE

Ek orchestrator/service jo steps chalata hai, state rakhta hai, har step ka result log karta hai — framework kaunsa hoga wo baad ka decision hai.

request
  -> input guardrail (L9.6)
  -> cache lookup (L9.5)   -- hit? respond
  -> route (L9.3)          -- kaunsa model / approach
  -> retrieve (L9.2)       -- knowledge laao
  -> model call            -- generate
  -> output guardrail + validate
  -> log + trace (L9.8)

Edge cases hi bugs: Flow ka rule: har arrow par decide karo kya hota hai agar ye step fail ya skip ho. Cache miss, empty retrieval, validation fail — missing edges hi production bugs hain.

Deterministic vs model-delegated decisions

Deterministic rakho: routing rules, schema validation, ACL, cache keys, retries, budgets — testable, saste, predictable.
Model ko chhodo: summarization, extraction ka judgment, ambiguity resolve, natural phrasing — jahan flexibility hi chahiye.
Rule: jitna deterministic, utna sasta aur eval-able. Model ka decision log karo taaki replay ho sake.
Model-delegated par contract: output schema + validation + fallback (L9.6) — bina uske model ka decision bina safety net ke hai.

Sawaal interview mein aata hai: "Why is this an LLM call?" — har model call ka jawab hona chahiye. Agar answer "fixed rule se ho jaata" hai toh wo code mein jaana chahiye.

Sync, async, queue ya long-running

Sync: interactive chat/autocomplete — sab kuch latency budget ke andar, warna user wait karega.
Async job: lamba kaam (bulk analysis, report) — accept + job id + status/poll, turant wapas aao.
Queue:
Long-running + checkpoints: agent runs — har step state persist (L6 notes), crash par resume.

Decision rule: Kya user turant jawab maangta hai? Haan → sync (budget tight). Nahi → queue/async. Agent ka har step bhi sync hai par poora run async chalta hai — dono baatein alag hain.

Component boundaries — alag-alag badalne layak

Har component ka apna boundary contract ho: input schema, output schema, failure behaviour aur metrics. Isi se parts alag-alag test, evaluate aur scale ho sakte hain (per-component evals L7, per-component metrics L10).

Retrieval ko replace kiya, contract same — generator ko pata bhi nahi chalega.
Ek component ka naya version rollback karo toh doosre unaffected rahein.
Anti-pattern: agentic complexity — har step ko agent bana dena. Steps count karo, unme se kitne model decisions hain? Agar sab fixed order hai toh simple workflow hi chalo.

Workflow first: Pehle fixed workflow likho, model calls sirf jahan zaroori. Agent tab jab plan khud banana ho — "interesting lagta hai" design justification nahi hai.

Exercise

Question: Rules, validation aur ACL jaise fixed steps — ek shabd, English?

Question: Lamba kaam accept karke background mein chalane ka pattern — do shabd, English?

Question: Bursty load absorb karne wala buffer — ek shabd, English?

Question: Har step ko agent bana dene wali soch — do shabd, English?

Common mistakes

Model se deterministic kaam karwana: ACL, schema check, routing rule — sab model par chhod diye toh mehnga, unstable aur test-layak nahi. Ye sab code mein likho, model sirf sochne wale kaamon ke liye.
Sab kuch sync rakhna: Lamba kaam sync mein = latency budget todta hai aur connection ghutti hai. Accept + queue + status — user ko turant wapas milega.
Boundaries bina contract ke: Components directly dusre ke internals par depend toh har change poora system hilata hai. Input/output schemas define karo — tabhi parts independently evaluate honge.

Desi example: Railway junction — orchestration ka desi analogy

Junction par schedule deterministic hai — fixed order, pehle se likhe rules; sirf exceptions par signalman decide karta hai. Pipeline wahi: zyada tar steps fixed, sirf wahi jagah model:

fixed: guard -> cache -> route -> retrieve -> generate -> check
model: sirf wahi step jahan "sochna" chahiye
rule : har step ka input/output contract = signal rule book

Interview angle — yahan se sawaal aate hain

Q: Ek request ka end-to-end flow kaisa hoga? (guardrail → cache → route → retrieve → generate → validate → log)
Q: Kaunse decisions deterministic rakhoge? (routing, validation, ACL, retries, budgets — model ko sirf judgment wale)
Q: Sync vs async kab chunoge? (user wait kare → sync; lamba/batch → async job/queue; agent run → checkpoints)
Q: Component boundaries kaise define karoge? (schemas, failure behaviour, metrics — independent eval/rollback ke liye)

Lesson complete?

Flow clear. Agla lesson — Monitoring, Feedback & Continuous Improvement: chalne ke baad system ko observe aur improve karna.