Lesson 45 — Beginner

Prompt Versioning, Cost & Caching

Prompt bhi code ki tarah versioned asset hai — badalta rehta hai, kisi ne kuch gadbad ki toh trace hona chahiye. Is lesson mein hum dekhenge prompts ko git ki tarah kaise track karein, variants ko quality/latency/token/cost pe kaise compare karein, unnecessary context kaise kaatein, aur prompt caching ka faayda kaise uthaayein.

Prompts ko code ki tarah version karo

prompts/
  triage/
    v1.md   # baseline — 2/3 test pass
    v2.md   # few-shot examples add kiye -> 3/3
    v3.md   # JSON keys fix kiye, token 20% kam
    current.md -> v3.md

# Har file ke upar header:
# model: <tumhara model>   changed: 2026-10-12   test-score: 3/3
# changelog: v2 se farq — examples add kiye, urgency label fix kiye

Har production result ke saath prompt version + model name log karo. Warna "pehle toh theek chalta tha" wali debugging kabhi khatam nahi hoti — tumhe pata hi nahi hoga kaunsa prompt tha jo user ko dikh raha tha.

Quality, latency, tokens, cost — charon dekho

import time

t0 = time.time()
resp = call(prompt, ticket)
latency = time.time() - t0
in_tok, out_tok = resp.usage.input_tokens, resp.usage.output_tokens
print(f"latency={latency:.1f}s  in={in_tok}  out={out_tok}")

# Compare: v3 ka score 3/3, v2 ka 2/3 -> v3 jeeta (quality first).
# Score same ho toh tie-break: kam tokens ya kam latency wala version.

• Quality pehle: test-set score sabse important metric hai.
• Latency: user-facing chat mein lamba prompt (bade examples, purani history) dikhna mushkil bana deta hai.
• Tokens: input + output dono count hote hain — har request ke input mein jaane wale tokens seedha bill pe padte hain.
• Provider ka pricing page khud check karo — rates badalte rehte hain, purane numbers pe bharosa mat karo. Comparison ke liye token count hi enough hai.

Unnecessary context kaato

• Purani history har request mein bhej rahe ho? Sirf last N turns ya relevant summary bhejo.
• Same rules har baar dobara likh rahe ho? System prompt mein ek baar likho, user message mein repeat mat karo.
• Model ko kabhi na dikhne wale internal notes (user IDs, debug logs) bhejna band karo — waste bhi aur leak bhi.
• Kaato waqt test set zaroor chalao — context trimming mein aksar ek useful clue chala jaata hai aur score gir jaata hai.

Prompt caching aur reusable prefixes

[STABLE PREFIX — har request mein same]  <- caching ka faayda yahin milta hai
- system rules
- tool definitions / schema
- few-shot examples

[VARIABLE TAIL — sirf yahi badalta hai]
- aaj ka user ticket
- current conversation turn

Bahut se LLM providers prompt caching support karte hain: agar tumhara prompt ka prefix pehle se gaya hai, toh provider us hisse ko dobara process nahi karta — input ke ush hisse ka rate (aur aksar latency) kam ho jaata hai. Faayda tabhi milta hai jab prefix bilkul byte-for-byte same ho.

Exercise

Common mistakes

• Prompt ko file-system ki tarah track na karna: Prompt codebase mein hai toh git mein track karo — commit message mein kya badla aur kyun. Warna production result ke peeche ka prompt version kabhi pata nahi chalega.
• Har request mein wahi boiler_context bhejna: Same system rules, tools aur examples har baar bhej rahe ho — bina caching ke har baar poore tokens count hote hain. Stable prefix upar rakho taaki caching lage.
• Sirf quality dekh kar cost/latency ignore karna: Do prompts ka score same hai toh token/latency wala version chuno. Har request mein chhupa hua extra context user bill aur speed dono pe padta hai.

Register (git) :
  v1 : baseline list         -> bill #12 test-pass
  v2 : GST alag column       -> bill #18 test-pass
  v3 : 10 item se upar discount -> bill #25 test-pass

Kal complaint aaye "bill galat tha" -> v2 wali file kholo, diff dekho, turant pata chalo.