Lesson 106 — Advanced

Reliability, Guardrails & Graceful Degradation

Model, provider, tool, retrieval — koi bhi ek pal mein fail ho sakta hai. Design ka kaam hai pehle se decide karna: timeout kahan, retry kis budget se, guardrail kis boundary par, aur system kitna chota hoke bhi useful rahe.

Failure matrix — kya toot sakta hai

• Provider outage/slow: timeout + fallback (L9.3) + circuit breaker.
• Rate limits (429): Retry-After respect, per-key counters, queue karke bhejo.
• Malformed output: schema validation fail → repair retry → phir degrade.
• Tool failures: tool error ko chhota structured result banao model ke liye — crash nahi.
• Retrieval failure: zero results ya index down → "pata nahi" wala jawab, guess nahi.

Timeouts, retries, breakers, queues

def call_with_policy(fn, budget):
    for attempt in range(budget.retries):
        if breaker.open:                     # provider ko chance hi mat do
            return fallback(fn)
        try:
            return fn(timeout=budget.timeout)  # har call ka apna timeout
        except (Timeout, RateLimited) as e:
            sleep_backoff(attempt, retry_after=getattr(e, "retry_after", None))
    return fallback(fn)                       # budget khatam -> degrade

• Timeout: har call par compulsory — bina timeout ke ek provider poora worker le kar baith jaayega.
• Retry budget: finite attempts + backoff + jitter — infinite retry se provider par attack ban jaata hai.
• Circuit breaker: failures threshold par call band, probe se recover — healthy provider ko faayda.
• Queue: bursts absorb, retries ko order do, rate limits se bachao.

Guardrails — boundaries par, model pe nahi

• Input boundary: injection patterns, PII aur size check — deterministic, model ke bharose nahi.
• Output boundary: schema/format validation, numeric sanity, allow-list of sources.
• Action boundary: irreversible actions par approval gate + dry-run mode.
• Guardrail bhi degrade ho sakta hai: filter service down toh strict default (block/safe answer) — silent pass kabhi nahi.

Graceful degradation — ladder pehle se likho

level 0 : full answer (normal path)
level 1 : cached / precomputed answer (L9.5)
level 2 : chhota model, reduced scope
level 3 : reduced features (summary only, koi action nahi)
level 4 : human escalation (ticket + context)
level 5 : explicit "abhi complete nahi ho sakta" (fake confidence nahi)

• Har level ka trigger pehle likho — breaker state, validation failure count, budget breach. Aankhon se nahi.
• Har level ka owner bhi: kaun dekhega, kaun decide karega wapas level 0 par aane ke liye.
• User ko batao level badalne par (thoda slower, features reduced) — chhupana trust todta hai.
• Log karo kaun sa level kab trigger hua — ladder ki frequency hi reliability report hai.

Human approval aur service-level expectations

• Irreversible actions: refund, delete, payment, publish — human approval mandatory. Reversible (undo available) = auto chal sakta hai.
• Approval ko context do: kya hoga, kyun, evidence ke saath — sirf "approve?" button se reviewer ka time waste hota hai.
• SLOs likho: availability, latency aur quality ke targets, phir unhe measure karo (L9.8). "Achha chalega" koi SLO nahi.
• Error budget: target se zyada problems ho gayin toh naye features freeze, reliability par kaam.

Exercise

Common mistakes

• Sirf model pe trust: Prompt wala instruction nudge hai, guarantee nahi. Safety aur format checks boundaries par deterministic lagao — model badalta hai, gate nahi.
• Retry budget nahi: Fail → turant retry, baar-baar — provider par thundering herd, latency explosion. Finite retries + backoff + breaker, sab ek policy mein.
• Silent degradation: System chupke se chhota ho gaya, user ko pata hi nahi — trust ka sabse tez tootna. Level change par explicit message + log mandatory.

normal    -> full power
power cut -> inverter (cache)
load zyada-> chhote appliances (chhota model)
last      -> "bijli aane tak ruko" (explicit inability)