Lesson 151 — Advanced

Distillation

Distillation ka funda: ek bade, mehenge model (teacher) ke outputs se ek chhote model (student) ko train karo, taaki ek specific kaam chhota model bhi achhe se karne lage — kam latency, kam cost aur khud host karne ki aazaadi ke saath. Is lesson mein hum teacher→student pipeline, outputs ka generation aur filter, capability-for-cost trade-off, copied mistakes ke risks aur terms of use dekhenge.

⏱ 10 min• Advanced• Reinforcement Fine-Tuning (RFT)

Teacher to student — ye kaise chalta hai

Kya hai?

Distillation matlab bada model (teacher) ko real traffic jaise inputs pe outputs generate karwana, aur wahi outputs chhote model (student) ke training data mein daalna. Student sirf ek narrow kaam sikhata hai — poora teacher nahi banta.

Kab kaam aata hai?

Jab task chhota aur clearly defined ho, teacher ka output achha ho, aur latency/cost/self-hosting ka farq asli ho. Har request pe bada model bulaana justified nahi, par uska jawab chhota model de sakta hai.

Kahan milta hai?

Tumhare apne pipeline mein: generation script + filter + training run. Teacher koi bhi bada model ho sakta hai — hosted API wala ya tumhare infra ka bada open-weights wala.

Mechanism simple hai: student teacher ke weights dekh kar nahi, uske outputs dekh kar seekhta hai. Inputs tumhe chunne hote hain, outputs filter karne hote hain, aur phir wahi data SFT jaisi training mein jaata hai. Jitna teacher ka output tumhare asli kaam ke karib hoga, student utna hi us kaam pe strong hoga.

Scope yaad rakho: Student poora teacher nahi ban paata — sirf jis narrow workload pe train hua usmein strong hota hai. Pehle decide karo ki kaunsa ek kaam chhote model ko chahiye, warna "student bhi teacher jaisa hai" ki expectation le kar deploy karoge aur off-task prompts pe gir jaayega.

Teacher outputs — generate aur filter

# pipeline: sample inputs -> generate -> filter -> train
rows = []
for item in sample_real_traffic(inputs, n=2000):
    out = teacher.generate(item["prompt"], temperature=0.7)
    if eval_suite.passes(out, item):          # filter training se PEHLE
        rows.append({"prompt": item["prompt"], "response": out})

# diversity ke liye: real prompts + paraphrases + edge cases
# seedha har output train mat daalo — ganda teacher = ganda student

Inputs real traffic jaise hon: teacher ko usi tarah ke prompts pe chalao jaise production mein aayenge — sirf synthetic shabd nahi.
Filter with your evals: teacher bhi galat karta hai. Jo output tum apne eval suite se pass nahi karate, wo student ke liye gold kaise banega?
Diversity maintain karo: paraphrases, edge cases aur alag-alag phrasings rakho — warna student ek hi template pe narrow ho jaayega.
Style bhi filter karo: teacher ka lamba, ghuma-phira hua style pasand nahi aata toh wo bhi dataset se hata do — student wahi copy karega.

Student ko bhi eval pe rakhho: Teacher ke held-out inputs pe student ko chalao aur compare karo — kitna gap bacha. Ye gap hi tumhara "capability given up" ka number hai, jo cost saving ke saath likhna padta hai.

Capability chhodo, cost-latency lo

TEACHER (hosted / bada)        STUDENT (self-hosted / chhota)
cost per request:   mehnga               sasta
latency:            network + bada model chhota, apne box pe
quality (task):     reference — isi pe measure karo
quality (off-task): strong               kam — dhyan do
control:            provider terms       tumhare GPU

har request pe note karo: task score, p50/p95 latency,
cost per 1k requests — teeno side-by-side

Asli win: thoda capability chhod kar cost, latency aur self-hosting milna — jo chhote task pe aksar kaafi hota hai.
Measure both sides: kitna quality gira (capability given up) aur kitna paisa/bachat mili (cost saved) — dono likho, warna faisla adhoora hai.
Router pattern: pehle student se pucho, agar confidence ya format fail ho toh teacher ko fallback bana do — dono ka best.
Student ka model card khud banao: kaunsa task supported hai, kaunsa nahi — serving team isi se handle karegi expectations.

Decision rule: Jab traffic itna high ho ki har request pe bada model ka kharcha ya latency budget toda jaaye, tabhi distillation karo. Chhote traffic pe toh seedha bada model hi sasta padta hai — student banane ka kharcha bhi toh hai.

Risks — copied mistakes, low diversity aur terms

Copied mistakes: teacher ki har galti training data ban jaati hai — student wo bina samjhe repeat karega. Filter hi ek hi bacha hua hai.
Low diversity: ek hi phrasing ke hazaaron examples = student ko lagta hai duniya wahi hai. Naye input patterns pe wo fail hoga.
Drift: teacher outputs ek time ke hain; production inputs badal jaayein toh student ke paas uska exposure hi nahi.
Terms of use: kuch providers apne model outputs ko competing model train karne se restrict karte hain — generate karne se pehle terms padho, baad mein legal notice nahi chahiye.

pehle check (training run se pehle):
1. teacher ki terms: outputs -> training allowed?
2. output filter: eval pass + diversity check
3. student eval: teacher ke held-out pe gap note karo
4. off-task sample: kya student general abilities khoyi?

in 4 ke bina distillation chalani = andhere mein chalana

Terms pehle, pipeline baad mein: Ek baar provider ne terms badal di toh pura pipeline dobara chalana pad sakta hai. Teacher choose karte waqt hi likh lo ki outputs training ke liye allowed hain — ye ek line aadhe project ke liye risk khatam kar deti hai.

Exercise

Question: Bada model teacher, chhota model student — ye training process kya kehlata hai?

Question: Teacher outputs ko training data banane se pehle kis cheez se filter karte hain?

Question: Student teacher ki galtiyan bina samjhe repeat kare — is distillation risk ko kya kehte hain?

Question: Provider ke outputs se competing model train karna kabhi kabhi banned hota hai — training se pehle kya padhna zaroori hai?

Common mistakes

Bina filter ke teacher outputs train mein daalna: Teacher ki galtiyan aur lamba style seedhe student mein chale jaate hain. Fix: har output ko eval suite se pass karao, diversity sample lo, tabhi dataset mein rakho.
Ek hi phrasing ke examples bhar dena: Low diversity se student narrow ho jaata hai — naye input pattern pe fail. Fix: real prompts + paraphrases + edge cases ka mix banao.
Teacher terms of use na padhna: Kuch providers outputs se competing model train karne se rokte hain — late pata chalta hai. Fix: teacher choose karte waqt hi training clause check karo.
Capability gap na naapna: Student deploy kar diya par off-task pe gir gaya — kya khoja ye number hi nahi tha. Fix: teacher ke held-out pe student chalao, gap likho, usi ke saath cost saving justify karo.

Desi example: Topper ke notes se chhote batch ki taiyari

Coaching mein chhote batch wale topper ke solved notes se padhte hain — saare answers wahi hain, par notes ki apni chhutiyon (galtiyan) bhi wahi chhap jaati hain, aur jo type question topper ne kabhi nahi likha wo chhote batch ko aata hi nahi:

topper (teacher) -> solved notes generate
filter (evals)   -> galat solutions hatao
chhota batch     -> notes se padho (student train)
naya paper       -> sirf notes wale type solve (scope)

notes ki galti = student ki galti (copied mistakes)

Interview angle — yahan se sawaal aate hain

Q: Distillation aur normal fine-tuning mein kya farq hai? — Distillation ka data khud bada model deta hai; normal fine-tuning mein tum apne expert/log examples dete ho.
Q: Teacher outputs ko seedha train set mein kyun nahi daalte? — Teacher bhi galat karta hai; eval se filter kiye bina galtiyan aur style seedhe student mein jaati hain.
Q: Distillation ka main trade-off kya hai? — Thoda capability chhod kar cost, latency aur self-hosting milta hai — dono taraf ka number measure karna part hai.
Q: Student kitna strong banega? — Sirf jis narrow workload pe train hua utna; off-task abilities aksar kam rehti hain — isliye scope likh ke deploy karte hain.
Q: Provider ke outputs se model train karte waqt kya check hota hai? — Terms of use: kuch providers competing-model training ko restrict karte hain.

Lesson complete?

Distillation clear — ab tum teacher→student transfer ko filter, measure aur terms ke saath chala sakte ho. Agla lesson — Evaluating Fine-Tunes — mein dekhenge tune ka value kaise prove kiya jaata hai, ya honestly reject kiya jaata hai.