Learn�GenAI�Fine-tuning

Lesson 07 � Intermediate

FINE-TUNING SE
MODEL APNA BANAO.

Pre-trained model ko apne data pe train karke custom behavior aur domain knowledge add karo. Prompt engineering se aage badho aur model ko genuinely apna banao.

? 22 min✓ Intermediate✓ Prerequisite: Prompt Engineering

WHY: Fine-tuning kyun zaroori hai?

Prompt engineering se bahut kuch ho sakta hai, lekin kai baar model ko apne specific domain ka knowledge dena hota hai � medical, legal, ya company-specific formatting. Fine-tuning se pre-trained model ko apne data pe train karke usme custom behavior add karte hain.

TRANSFER LEARNING

Pre-trained model ka knowledge lena aur apne task pe adapt karna. Yeh fine-tuning ka foundation hai � zero se train karne ki zaroorat nahi.

LoRA

Low-Rank Adaptation � sirf chhote adapters train karte hain, pura model freeze rakhte hain. Kam compute, same quality. Industry standard approach.

RLHF

Reinforcement Learning from Human Feedback � human preferences se model ko aur better banate hain. ChatGPT isliye itna achha hai.

DATA PREP

Training data ki quality sabse important hai. Garbage in = garbage out. Data clean karna, format karna, split karna � yeh 80% kaam hai.

WHAT: Fine-tuning kya hai?

Fine-tuning ek aisi technique hai jismein hum ek already trained model ko additional data ke saath thoda aur train karte hain. Isse model naya skill seekhta hai ya apne behavior ko change karta hai.

ℹ
Mental model: Fine-tuning ek experienced cook ko nayi recipe sikhane jaisa hai. Usse basic cooking aati hai (pre-training), ab aap usse apni specific dish ki technique sikha rahe hain (fine-tuning).

APPROACHES: Fine-tuning ke tareeke

Alag-alag situations ke liye alag approaches hain. Sabse pehle samjho kab kya use karna hai:

comparison
 Full Fine-tuning:
 � Poora model update hota hai
 � Zyada GPU memory chahiye
 � Best jab: large dataset + unique behavior
 � Cost: $$$

 LoRA (Low-Rank Adaptation):
 � Sirf chhote adapters train
 � 90% kam compute
 � Best jab: limited resources
 � Cost: $

✓ QLoRA:
 � LoRA + 4-bit quantization
 � Even kam memory
 � Best jab: single GPU pe kaam karo
 � Cost: $

 RLHF:
 � Human feedback se align
 � Safety + helpfulness badhao
 � Best jab: chat/instruction models
 � Cost: $$

HOW: Fine-tuning pipeline kaise banti hai

Fine-tuning ka process step-by-step hai. Har step important hai:

pipeline
Step 1: DATA PREPARATION
 ✓ Training data collect karo (JSON, CSV, text)
 ✓ Format karo (instruction-input-output format)
 ✓ Train/validation split karo (80/20)

Step 2: MODEL SELECTION
 ✓ Base model choose karo (GPT-2, Llama, Mistral)
 ✓ Size decide karo (1B, 7B, 13B parameters)
 ✓ Quantization level choose karo (FP16, INT8, INT4)

Step 3: TRAINING CONFIGURATION
 ✓ Learning rate set karo (1e-4 to 5e-5)
 ✓ Batch size decide karo
 ✓ Epochs set karo (1-5 typically)
 ✓ LoRA config setup karo

Step 4: TRAINING
 ✓ Model ko data pe train karo
 ✓ Loss monitor karo
 ✓ Validation check karo
 ✓ Best checkpoint save karo

Step 5: EVALUATION
 ✓ Test set pe evaluate karo
 ✓ Quality check karo
 ✓ Deployment ke liye ready karo

CODE: HuggingFace se fine-tuning

Ab actual code dekhte hain. HuggingFace library se fine-tuning banana kitna easy hai:

python
# Fine-tuning with HuggingFace
from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments, Trainer
from datasets import Dataset

# Sample training data
data = {
 "text": [
 "Q: What is DSWallah✓ A: DSWallah is a learning platform.",
 "Q: How to learn Python✓ A: Start with basics, practice daily.",
 ]
}
dataset = Dataset.from_dict(data)

# Load model
model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Tokenize
def tokenize(examples):
 return tokenizer(examples["text"], truncation=True, padding=True)

tokenized = dataset.map(tokenize, batched=True)

# Training
training_args = TrainingArguments(
 output_dir="./results",
 num_train_epochs=3,
 per_device_train_batch_size=2,
)

print("Fine-tuning setup complete!")
print("In production, use: Trainer(model=model, args=training_args, train_dataset=tokenized)")
ℹ
Key insight: Yeh simplified example hai. Real production mein data preparation, LoRA configuration, aur evaluation metrics bahut important hain. Pehle concept samjho, phir complex implementation karo.

LoRA: Efficient fine-tuning ka king

LoRA sabse zyada use hone wala technique hai kyunki yeh kam compute mein achhe results deta hai. Dekhte hain kaise kaam karta hai:

python
# LoRA setup with PEFT library
from peft import LoraConfig, get_peft_model

# LoRA configuration
lora_config = LoraConfig(
 r=8, # Rank - kitne naye parameters
 lora_alpha=32, # Scaling factor
 target_modules=["q_proj", "v_proj"], # Kaunse layers modify
 lora_dropout=0.1, # Regularization
 bias="none", # Bias train nahi karna
)

# Model pe LoRA apply karo
model = get_peft_model(model, lora_config)

# Parameters ki summary
model.print_trainable_parameters()
# Output: trainable params: 471,859 || all params: 124,439,808 || 0.38%

Dekha✓ Sirf 0.38% parameters train ho rahe hain lekin model ka behavior change ho raha hai. Yeh hai LoRA ka magic.

DATA PREP: Training data kaise banaye

Data preparation sabse important step hai. Agar data achha nahi hai toh model bhi achha nahi banega.

python
# Training data format examples
training_data = [
 {
 "instruction": "Summarize the following text",
 "input": "Python ek programming language hai jo...",
 "output": "Python ek high-level programming language hai."
 },
 {
 "instruction": "Translate to Hindi",
 "input": "Hello, how are you?",
 "output": "Namaste, aap kaise hain?"
 },
 {
 "instruction": "Write code for",
 "input": "factorial of a number",
 "output": "def factorial(n):\n if n == 0:\n return 1\n return n * factorial(n-1)"
 }
]

# Data quality checklist:
# 1. Format consistent hai?
# 2. Diverse examples hain?
# 3. Errors ya typos nahi hain?
# 4. Train/validation split kiya?
# 5. Token limits ke andar hai?
ℹ
Pro tip: Data preparation mein 80% time lagta hai. Jitna zyada clean aur diverse data hoga, utna achha model banega. Quality > Quantity.

Try it: Fine-tuning pipeline setup karo

Neeche apni training configuration set karo aur pipeline ka output dekho:

Fine-tuning Pipeline BuilderApna configuration set karo
Configuration set karo aur Build Pipeline dabao
Demo mode � Real training ke liye GPU chahiye

Exercise

Fine-tuning aur transfer learning mein kya fark hai✓ Ek line mein explain karo.

Socho: Transfer learning ek broad concept hai � pre-trained model lena. Fine-tuning uska specific implementation hai � apne data pe train karna.

WHEN: Kab fine-tuning karna chahiye

Fine-tuning hamesha zaroori nahi hota. Pehle samjho kab karna chahiye:

decision
✓ FINE-TUNING KARO JAB:
 ✓ Prompt engineering se kaam nahi chal raha
 ✓ Domain-specific knowledge chahiye
 ✓ Consistent format/behavior chahiye
 ✓ Large, high-quality dataset hai
 ✓ Compute budget hai

✓ FINE-TUNING MAT KARO JAB:
 ✓ Prompt engineering se ho raha hai
 ✓ Data kam hai (< 100 examples)
 ✓ Sirf formatting change karni hai
 ✓ RAG se kaam chal sakta hai
 ✓ Budget limited hai
ℹ
Career tip: Fine-tuning ab bahut demand mein hai. Healthcare, legal, finance industries ko domain-specific models chahiye. Yeh skill seekho toh senior AI engineer ban sakte ho.

TOOLS: Fine-tuning ke popular tools

Industry mein kaunse tools use ho rahe hain:

ecosystem
 HuggingFace Ecosystem:
 � transformers � Model loading aur training
 � peft � LoRA, QLoRA implementation
 � datasets � Data loading aur processing
 � trl � RLHF training

 Cloud Platforms:
 � OpenAI Fine-tuning API � easiest option
 � Google Vertex AI � enterprise scale
 � AWS Bedrock � managed fine-tuning
 � Azure ML � enterprise with compliance

 Specialized Tools:
 � Unsloth � Fast LoRA training
 � Axolotl � Multi-method training
 � LLaMA Factory � Easy fine-tuning
 � MLX � Apple Silicon optimized
Fine-tuning lesson complete?

Ab RAG par chalo � jo fine-tuning ka best alternative hai jab data large hai aur real-time chahiye.