Lesson 112 — Advanced

Preparing Your Apps for Production

Production ka matlab hai "kal bhi yahi chalega aur koi bhi isse badal sake bina tode". Is lesson mein hum configuration ko code se alag karna, model aur dependencies pin karna, pehli request se traceability, aur Level 7 wale eval set ko fixed yardstick ki tarah aage le jaana dekhenge.

Taiyaari kyun zaroori hai

Har LLM app mein kuch aisi cheezein hoti hain jo behaviour badalti hain: chunk size, top-k, model, temperature, prompt text, tool list. Inhe code mein hardcode kiya toh har badlaav ek code change hai — review, test aur rollback mushkil. Config mein daala toh wahi badlaav ek chhota diff ban gaya.

Configuration, not code

from dataclasses import dataclass
import yaml

@dataclass
class AppConfig:
    model: str = "provider/model-name"
    temperature: float = 0.0
    chunk_size: int = 500
    top_k: int = 5
    max_agent_steps: int = 8
    prompt_version: str = "v3"

def load_config(path: str = "config.yaml") -> AppConfig:
    with open(path) as f:
        data = yaml.safe_load(f) or {}
    return AppConfig(**data)   # unknown key -> turant error

# config.yaml (repo mein versioned):
# model: provider/model-name
# temperature: 0.0
# chunk_size: 500
# top_k: 5
# max_agent_steps: 8
# prompt_version: v3

• Dataclass ya pydantic model banao — galat key ya galat type pe app shuru hone se hi reject ho jaayega.
• Env se sirf secrets aur deployment-specific cheezein aayein (API key, host); behaviour wali settings file mein — warna "mere laptop pe chala" wali problem.
• Config ka har value ka default wahi ho jo production mein chal raha hai — taaki config missing pe bhi system surprise na de.
• Config bhi git mein commit hoti hai — config change = reviewable diff, log ke saath.

Model aur dependency pinning

requirements.txt (pinned):
fastapi==<exact>
uvicorn[standard]==<exact>
# har dependency exact version pe — "next month ka build = aaj ka build"

model config:
model: provider/model-name@version
# "latest" kabhi mat likho — kal alag model milega

lockfile (poetry.lock / uv.lock) bhi repo mein:
# direct pins + transitive pins, dono reproduce hoti hain

• Dependency pins ke bina CI ka "green" kal red ho sakta hai — aaj install hi nahi hui wali version.
• Model ka version bhi pin, kyunki provider ka "latest" chupke se badal sakta hai — release ka wo bhi hissa hai.
• Pin upgrade ek dedicated change hai: naya pin + eval suite + note ki kya badla. Mixed "sab update kar dete hain" PR se bachein.

Pehli request se traceable

import uuid, time

def handle_request(user_msg: str) -> dict:
    rid = uuid.uuid4().hex[:12]          # request ID
    t0 = time.perf_counter()
    answer = rag_chain(user_msg)
    trace = {
        "request_id": rid,
        "prompt_version": CFG.prompt_version,
        "model": CFG.model,
        "config_hash": hash_config(CFG),
        "latency_s": round(time.perf_counter() - t0, 3),
    }
    log_trace(trace)                      # har step isi rid ke saath
    return {"answer": answer, "request_id": rid}

• Request ID sab kuch jodta hai: user ka sawaal, retrieved chunks, model call, guardrail blocks, final answer — ek ID se.
• Prompt version aur model har trace mein likho — warna "kaunse prompt se ye jawab aaya" ka sawaal answer hi nahi hoga.
• Config ka hash ya snapshot bhi store karo — top-k 5 tha ya 8, baad mein decide karna impossible hai.
• Ye overhead chhota hai (ek dict aur log line) — par debugging mein dinon bachata hai.

Level 7 ka eval set — fixed yardstick

Level 7 mein jo evaluation dataset banaya tha, usse ab fixed yardstick ki tarah le chalo: har change — code ho ya prompt ho ya model ho — isi pe naapa jaayega. Dataset ko repo mein versioned rakho, na ki kisi ke laptop pe.

evals/
  rag_cases.jsonl   # question + gold answer/source
  agent_cases.jsonl # task + expected tool sequence
  README.md         # kab add hua, kab retired hua, kyun

# rule: eval case delete nahi hota, retire hota hai (reason ke saath)
# naya production failure = naya case (L10.9 se aayega)

Exercise

Common mistakes

• Har cheez hardcode karna: Model, chunk size, top-k code mein likhe toh har badlaav ek code change aur deploy ban gaya. Fix: behaviour settings config file mein, code sirf logic.
• Unpinned dependencies: Aaj ka green build kal install hi nahi hui wali version pe red. Fix: exact pins + lockfile repo mein, upgrade ek dedicated reviewed change.
• Traceability baad mein add karna: Bina request ID ke production ki kisi bhi report ko reproduce karna impossible hai. Fix: pehli request se request ID, prompt version aur model har log line mein.

thela (notebook app)   -> sab chef ke mood pe
restaurant (prod app)  -> printed menu (config)
                         -> recipe card (pinned versions)
                         -> order slip no. (request id)
                         -> taste standard (eval set)