Lesson 146 — Advanced

Data Curation for Fine-Tuning

Fine-tune ki quality 90% data se decide hoti hai — model sirf wahi seekhta hai jo tum doge. Is lesson mein hum pehle behaviour spec likhna, examples ke sources aur unke bias, chat template match, loss masking, cleaning, balanced edge cases, leak-free splits aur dataset versioning ek ek karke dekhenge.

⏱ 20 min• Advanced• Should You Fine-Tune?

Data curation kyun sabse pehla kaam hai

Kya hai?

Fine-tuning dataset banana: har example ka input, sahi output, sahi format — aur saaf, balanced, versioned form mein. Model jo dekhega wahi banega — ganda data toh ganda behaviour, bahut mehnat se.

Kab kaam aata hai?

Training se pehle, aur training ke baad bhi — har naye failure case ke saath dataset update hoti hai. Dataset hi tumhara asli aset hai, weights toh uska nateeja hain.

Kahan milta hai?

Production logs se, experts ke dwara likhe examples se, filtered synthetic data se — phir JSONL files ke roop mein training scripts ko. Split aur manifest isi ke saath chalte hain.

Sabse aam galti ye hai ki log pehle tool khol lete hain aur data baad mein. Sahi order ulta hai: pehle behaviour ki specification likho, phir us specification ke against har example ko check karo. Iske bina har example pe individual judgement chalta rahega aur dataset mein 10 tonein mil jaayengi.

Spec pehle: Ek chhota SPEC object banao — kya chahiye, kya bilkul nahi chahiye. Har example isi spec se filter hoga. Ye 30 minute ka kaam aadhe dataset problems pehle hi khatam kar deta hai.

Examples kahan se aate hain

Real usage logs: asli traffic ke input + jo output tumne approve kiya. Sabse realistic, par ismein poor requests, PII aur historical mistakes bhi hote hain — filter zaroori.
Expert-written demonstrations: domain expert ne khud sahi jawab likhe. Sabse saaf aur consistent, par limited quantity — expert ka waqt mehnga hai.
Synthetic data: ek bade model se examples generate karke manually filter kiye. Tez aur scalable, par repetition, ek hi phrasing aur teacher ke blind spots inherit ho jaate hain.
Hybrid is the default: logs se coverage, expert se quality anchor, synthetic se quantity — teeno ka mix aksar sabse balanced ban ta hai.

# Har source ka bias alag hai — mix karo, blind mat raho
sources = {
    "log":     "realistic, par PII + purane ghalat jawab bhi hain",
    "expert":  "clean, par chhoti quantity, ek hi expert ki aadat",
    "synthetic": "bhar ke quantity, par phrasing repeat + teacher ki galtiyan",
}

# pipeline: harvest -> filter -> expert review -> version
rows = filter_against_spec(SPEC, harvest(logs, expert_written, synthetic))

Synthetic data ka rule: Jo model produce kar raha hai usse blind trust mat karo. Har synthetic batch ka sample padho — ek hi phrasing, ek hi structure har jagah dikhe toh student model bhi utni hi narrow ho jaayega.

Chat template match aur loss masking

# Ek training example (JSONL, har line ek example)
{"messages": [
  {"role": "system", "content": "Tum ek support assistant ho."},
  {"role": "user", "content": "Refund kab milega?"},
  {"role": "assistant", "content": "Order dekh kar 5-7 working days ka window batata hoon."}
]}

# base model ne jis chat template se training ki, wahi format yahan chahiye
# galat template -> model bhi wahi galat framing seekhega (silent degradation)

Template = base model ki aadat: har model ka system/user/assistant markup alag hota hai. Tumhara JSONL usi template mein convert hona chahiye jismein base model instruction-tuned hua tha.
Loss sirf assistant tokens pe: system aur user ka text context hai, target nahi. Masking se training sirf "jawab kaise dena hai" se seekhti hai — warna prompt dohonchhna bhi learn hota hai.
Truncated responses hatao: beech kata hua jawab model ko incomplete answer ki aadat deta hai. Sequence limit se chhote examples hi rakho.
System prompt decide karo: production mein system prompt doge toh training examples mein bhi wahi ho — dono jagah alag toh behaviour alag.

Template mismatch = sabse common silent bug: Model chal jaata hai, output bhi aata hai, par quality expectations se kam — kyunki template alag tha. Pehle 20 examples ka rendered form nikal ke aankh se padho, tabhi training start karo.

Cleaning — thode saaf, hazar gande se behtar

import hashlib
from collections import defaultdict

def norm(t):
    return " ".join(t.lower().split())

def audit(rows):
    groups = defaultdict(list)
    for ex in rows:
        groups[norm(ex["prompt"])].append(ex)

    dup = sum(len(g) - 1 for g in groups.values())
    contradictions = 0
    for g in groups.values():
        outs = {norm(e["assistant"]) for e in g}
        if len(outs) > 1:            # same input, alag-alag jawab = label fight
            contradictions += 1
    return dup, contradictions        # dono theek karke hi aage badho

Duplicates: same example 5 baar train set mein toh model usi pattern pe over-fit ho jaata hai — count check karo.
Contradictions: ek hi input ke do alag jawab (refund 5 din bhi, 10 din bhi) — model confuse ho jaata hai. Decide karo kaunsa sahi hai, ek hi chhodo.
Wrong labels aur truncated outputs: manually review ka sample lo, galat nikal ke phenko.
Balance: 90% easy cases + 10% hard toh model easy hi sahi maanega. Edge cases ko deliberately oversample karo.
Few clean beats many noisy: narrow task pe kuch saafe representative examples hazaron noisy se zyada kaam karte hain.

Balancing ka test: Dataset ka random sample nikaalo aur kaho — "ye 20 mein se kitne hard cases hain?" Agar jawab lagbhag zero, toh model hard case kabhi nahi seekhega. Har important edge case ka apna quota rakho.

Leak-free splits, PII aur versioning

import hashlib, json

def norm(t): return " ".join(t.lower().split())
def bucket(t): return int(hashlib.sha256(norm(t).encode()).hexdigest()[:8], 16) % 100

# split fingerprint pe, taaki near-duplicate ek hi side rahe
for ex in rows:
    b = bucket(ex["prompt"])
    ex["split"] = "test" if b < 10 else ("val" if b < 20 else "train")

manifest = {
    "name": "support-v3",
    "rows": len(rows),
    "spec": SPEC,
    "counts": count_by_split(rows),
}
# manifest + data dono version control mein — yahi traceability hai

Teen splits: train (seekhne ke liye), val (hyperparameter aur checkpoint ke liye), test (ekdum last mein, ek baar chalane ke liye).
Leak-free matlab near-duplicate bhi ek hi side: same order ka 2 version split ke alag-alag hisson mein gaya toh test set ka score jhooth bolega.
PII check: phone, email, account numbers model ke weights mein baith sakte hain — scan karo, mask ya hatao.
Licensed content: jis content ka license training allow nahi karta, wo dataset mein aana nahi chahiye. Data source ki terms pehle padho.
Versioning: har run ke saath dataset ka naam + hash + config save karo — tuned model kis data se bana, ye baad mein zaroor poochha jaata hai.

Do not skip the held-out test: Validation baar baar dekhoge, usse model ki aadat set ho sakti hai. Asli test set ek baar — final decision ke waqt. Aur test set har naye dataset version pe naya nahi hota; wo tumhara stable judgements ka pen hai.

Exercise

Question: Data collect karne se pehle sabse pehle kya likha jaata hai jisse har example check hota hai? (ek shabd)

Question: Training loss sirf kis cheez ke tokens pe honi chahiye — prompts ya assistant responses?

Question: Train, val aur test ke alag-alag hisson mein same near-duplicate example chala jaaye — us problem ko kya kehte hain? (ek shabd)

Question: Dataset ko training configuration ke saath saath kis cheez mein track karte hain taaki har model trace ho sake?

Question: Ek hi input ke do alag jawab dataset mein hon — is inconsistency ko kya kehte hain?

Common mistakes

Bina spec ke data collect karna: Har example pe individual judgement chalta rahega, dataset mein das ton mil jaayengi. Fix: pehle behaviour spec likho, phir har example usi se filter karo.
Galat chat template mein examples daalna: Markup alag hua toh tune silently degrade hota hai — chal jaata hai par bekaar. Fix: base model wale template mein render karke pehle 20 examples padho.
Synthetic data ko blind trust karna: Repeat phrasing aur teacher ki galtiyan seedhe student mein jaati hain. Fix: sample padho, dedupe karo, spec se filter karo — quantity kabhi quality ka substitute nahi.
Test split tak pahunchte hi leakage: Near-duplicate do taraf gaya toh test score inflated — tune ka faisle hi galat. Fix: split fingerprint (normalized input hash) pe karo, row index pe nahi.

Desi example: Aate ko chhaanna — chhanni pehle, kadai baad mein

Halwai sabse pehle aata chhaantaa hai — patthar, kachra, gandagi alag — tabhi kadai lagti hai. Fine-tuning mein bhi: data saaf, balanced aur chhanna hua (splits) tabhi, warna jo ganda gaya wahi kadai mein se aayega:

kachra (duplicates/PII)   -> chhaan lo
patthar (contradictions)  -> nikaal do
ek hi size (balancing)    -> edge cases add karo
alag dabba (split)        -> train / val / test leak-free
recipe card (version)     -> manifest — kis data se bana

Interview angle — yahan se sawaal aate hain

Q: Dataset ke teen splits kyun hote hain? — Train seekhne ke liye, val tuning/selection ke liye, test final honest estimate ke liye.
Q: Loss masking kya hai aur kyun? — Sirf assistant tokens pe loss taaki model prompt dohonchhna na seekhe, sirf jawabarna seekhe.
Q: Chat template mismatch ka kya asar hai? — Silent quality drop: model chalta hai par base ki aadat se bahar framing mein train hua.
Q: 500 saaf examples vs 5000 shorile — kya chunoge? — 500 saaf, kyunki narrow task pe noise contradictions aur bad habits laati hai.
Q: Dataset versioning kaise karte ho? — Naam, row count, spec aur split counts ka manifest, har training run ke saath save.

Lesson complete?

Data pipeline clear — ab tum saaf, balanced aur versioned dataset bana sakte ho. Agla lesson — Supervised Fine-Tuning (SFT) — mein isi data ko actual training run mein badalna dekhenge.