Lesson 153 — Advanced

Serving the Adapter

Training sirf aadha kaam hai — tuned model ko production mein laana asli mushkil hai. Is lesson mein hum adapter vs merged weights ka choice, save/load/merge, multi-LoRA serving, local runtimes ke liye quantized export, template/tokenizer match, versioning aur rollback, gateway se gradual rollout, latency-cost comparison aur retraining triggers dekhenge.

⏱ 20 min• Advanced• Evaluating Fine-Tunes

Adapter serving kyun alag game hai

Kya hai?

Adapter serving matlab trained LoRA ko production endpoint pe laana — base model ke saath jod kar, bilkul wahi chat template aur tokenizer ke saath, versioned aur rollback-ready form mein.

Kab kaam aata hai?

Jab eval pass ho jaaye aur tune ko traffic dena ho — aur pehle se hi: jab ek base pe kai behaviour versions (customer, task, experiment) ek saath chalni hon.

Kahan milta hai?

Inference engines jo adapters load kar sakte hain, managed/serverless adapter hosting pe, ya local runtimes ke liye quantized export mein — har raaste ka apna trade-off.

Adapter serving ka core advantage: base model ek baar load hota hai, chhote adapters alag-alag behaviour dete hain. Isliye storage kam, swap tez, aur kai versions ek hi GPU pe. Par isi ke saath aata hai discipline — adapter, base, dataset, template sab pinned hone chahiye, warna production mein tune ka behaviour khokla ho jaata hai.

Serving = release engineering: Adapter ab ek release artifact hai jaise code ka build: versioned, tested, rollback wala. "Mere paas latest adapter hai wahi chala do" — production ka rule nahi hai.

Adapter separate ya merged weights

from peft import PeftModel   # example adapter library

# option A: adapter ALAG — base + adapter dono chahiye serving ke waqt
model = PeftModel.from_pretrained(base_model, "exports/support-v3")

# option B: MERGE — adapter weights base mein jud kar ek file
merged = model.merge_and_unload()
merged.save_pretrained("exports/support-merged-v3")

# merge ke baad eval dobara chalao — kabhi kabhi thoda farq aata hai

Adapter separate: storage chhota, swap second-level, multi-LoRA mumkin — par serving engine ko adapter support chahiye.
Merged weights: ek hi artifact, kisi bhi runtime pe chal jaata hai, ops simple — par har version ek poora weight copy, swap bhaari.
Merge verification: merged model ko usi held-out set pe chalao jo adapter ne pass kiya tha — numbers mile tabhi merge safe hai.
Decision rule: multi-LoRA ya frequent swaps chahiye → separate; ek stable version, generic runtime → merged export.

Merge = copy, adapter = reference: Adapter ko delete kar diya toh base akela bacha; merge kar diya toh dono ke beech ka link khatam. Storage planning mein farq — adapter rakhne mein sasta hai, isliye versions ki poori history bhi rakhi ja sakti hai.

Multi-LoRA hosting aur local export

# base model EK baar load, adapters naam ke saath attach (example shape)
vllm serve <base-model> \
  --enable-lora \
  --lora-modules support=exports/support-v3 billing=exports/billing-v2

# request mein adapter choose hota hai:
# POST /v1/chat/completions  {"model": "support", ...}
# GPU memory: base ek hi, adapters chhote + demand pe load

Multi-LoRA: ek base model pe kai adapters — per-customer ya per-task behaviour, bina har version ke liye alag server ke.
Adapter cache: pehli request pe load hoti hai, phir memory mein rehti hai — cold start ka dhyan rakho, health checks isi ke against likho.
Local runtime: merged model ko quantized format mein export karke local runtimes (Ollama jaisi) pe chalao — laptop/edge pe self-hosting yahin se aati hai.
Quantization ke baad eval: quantization tune ka kuch hissa undo kar sakta hai — export ke baad wahi held-out set phir se chalao, tabhi ship karo.

Template set pin karo: Adapter + base model + chat template + tokenizer — ye charon ek hi release bundle hain. Inmein se koi bhi alag version aa gaya toh output behaviour badal jaata hai, bina kisi error ke.

Template match, versioning aur rollback

ADAPTER RELEASE (har adapter ek release hai)
adapter:   support-lora-v3   (hash)
base:      <base-model>@<version>
dataset:   support-v3        (hash)
config:    r=16 alpha=32, epochs=3, lr=2e-5
template:  <wahi chat template jo training mein use hua>
eval:      reports/support-v3.pdf

rollback: v2 deployed hai -> gateway 100% v2 par palat do
          server restart nahi, sirf route badla

Template mismatch = production ka sabse common tune bug: model chalta hai par bekaar deta hai, kyunki serving template training wala nahi hai. Pehla check yahi.
Tokenizer bhi match: alag tokenizer se special tokens aur formatting bigadte hain — template ke saath hi pin karo.
Charon version saath: adapter, base, dataset, config — ek metadata file mein ek saath, warna "kis data se bana tha" baad mein dhoondha nahi jaayega.
Rollback ready: purana adapter deployable rakho. Naya version gadbad kare toh route palatna hi rollback — minutes mein.

Debug ka pehla sawaal: Tune production mein kharab chala? Lo, sabse pehle template/tokenizer check karo, phir base version, phir adapter hash. 90% baar mismatch yahin milta hai — training ko dosh dene se pehle.

Gateway rollout, latency-cost aur retraining triggers

ROLLOUT (gateway traffic split — bada jump nahi)
canary: 5% tuned / 95% baseline  -> errors, latency, judge score watch
50%                              -> logs mein behaviour diff padho
100%                             -> baseline ko rollback ke liye rakho

MEASURE vs prompted baseline: p50/p95 latency, cost per 1k, quality

RETRAIN TRIGGERS (agli version kab):
- naya base model available  - input distribution drift  - naye failure cases

Gateway hi tumhara traffic dial hai: base aur tuned ke beech split gateway pe rakho — deploy bhi wahin se, rollback bhi wahin se.
Canary ke metrics: error rate, latency percentiles, judge score aur production feedback — teenon chalne chahiye, sirf ek nahi.
Baseline se latency/cost compare: tune sirf quality se nahi jeetna, serving economics se bhi — per request numbers likho.
Retraining triggers pehle se likho: naya base model, drifting inputs, naye failure cases — kab agli version train karni hai, ye decide kiye hue hona chahiye.
Har naye version ko wahi eval suite gate kare jo pehle wale ko tha — special case mat banao.

Production feedback loop: User thumbs-down, edits aur support tickets — ye sab naye failure cases hain jo agli dataset mein jaayenge. Serving wahi hai jo is loop ko zinda rakhta hai: version → rollout → measure → retrain.

Exercise

Question: Adapter weights ko base model mein jod kar ek single artifact banana — us process ko kya kehte hain?

Question: Ek base model pe kai adapters ek saath serve karna — kya kehlata hai? (multi-LoRA / multi-model)

Question: Training aur serving mein bilkul match honi chahiye — template ke saath kaunsi cheez bhi?

Question: Traffic ko gradually base se tuned model ki taraf shift karwane wala layer kya hai? (ek shabd)

Question: Naya base model, input drift ya naye failure cases — ye sab kiske signals hain?

Common mistakes

Serving template ko training se alag rakhna: Model chalta hai par quality expectations se kam — sabse common silent bug. Fix: template + tokenizer ko adapter ke saath release bundle mein pin karo, deploy se pehle sample output padho.
Merge ya quantize karke bina eval ship karna: Dono tune ka kuch hissa chhupa sakte hain — number khud chup jaate hain. Fix: export ke baad wahi held-out set dobara chalao, tabhi release approve.
Sab traffic ek saath tuned model pe daal dena: Bug ho toh poora production affected, rollback bhi late. Fix: gateway pe canary (5% se shuru), metrics watch, phir badhao — purana version hamesha ready.
Adapter versions track na karna: Do hafte baad pata nahi kaunsa adapter kis base aur data pe chal raha hai — debugging impossible. Fix: adapter+base+dataset+config ka ek release manifest, previous version deployable rakho.

Desi example: Ek hi dal, alag tadke

Mess mein dal ek hi banti hai, har order pe usmein alag tadka lagta hai — kadi, jeera, lemon — aur koi bhi dabba palatne mein do minute nahi lagte. Multi-LoRA serving bhi yahi hai: base (dal) ek baar, adapters (tadke) request ke hisaab se, aur rollback = purana tadka wapas:

dal (base model)     -> ek baar bani, freezer mein (GPU)
tadka (adapter)      -> order pe lagta hai (request routing)
naya tadka (v3)      -> pehle 5% order pe trial (canary)
kharab laga?         -> turant v2 wapas (rollback)
recipe card          -> adapter+base+dataset+config manifest

Interview angle — yahan se sawaal aate hain

Q: Adapter alag rakho ya merge karo — kya decide karta hai? — Multi-LoRA/frequent swap chahiye toh separate (chhota, fast swap); generic runtime aur stable version ho toh merged (ek artifact).
Q: Multi-LoRA serving se kya milta hai? — Ek base load, kai adapters — per-customer behaviour bina har version ke alag GPU/server ke.
Q: Tune theek tha par production mein kharab chala — sabse pehla check? — Chat template aur tokenizer match, phir base version, phir adapter hash.
Q: Rollback kaise karte ho bina downtime ke? — Adapter versioned hai toh gateway route ko purane version pe palat do — sirf traffic badla, server nahi.
Q: Quantization ke baad kya karna zaroori hai? — Wahi held-out eval dobara chalana — quantization tune ka kuch hissa undo kar sakta hai.
Q: Agli tuning kab? — Retraining triggers se: naya base model, input drift, naye failure cases — har naye version ko wahi eval suite gate karta hai.

Lesson complete?

Serving clear — ab tum adapter ko versioned, rollback-ready aur measured form mein gateway ke peeche launch kar sakte ho. Agla module — Capstone — mein ye poora frame ek project mein jodoge: ek LoRA jo apne prompted baseline ko beat karta hai.