Lesson 09 — Data Science

CATEGORIES
KI PREDICTION.

Classification se categories predict karte hain — spam/not spam, pass/fail, cat/dog. ML ka sabse common task hai. Regression mein continuous value predict hoti hai, classification mein discrete label milta hai. Jaise doctor disease ka naam batata hai, waise classifier data ka category batata hai.

? 24 min✓ Intermediate✓ Prerequisite: Regression

WHY: Classification kyun zaroori hai?

Har jagah classification hai — email spam hai ya nahi, loan approve hoga ya nahi, patient sick hai ya healthy. Binary classification (2 categories) aur multi-class classification (3+ categories) dono hote hain. Decision Tree samajhna aasan hai, Random Forest zyada accurate hai, Logistic Regression baseline ke liye best hai. Real world mein ye algorithms sabse zyada use hote hain.

LOGISTIC REGRESSION

Binary classification ka sabse purana aur reliable algorithm. Sigmoid function se probability nikalta hai — 0 aur 1 ke beech. Linear regression jaisa hai lekin output probability hai, continuous value nahi.

DECISION TREE

Rules based classifier — tree jaisa structure banata hai. Har node pe ek question hai, har branch ek answer hai. Explain karna asaan hai lekin overfitting ka risk hai. Random Forest iska improved version hai.

RANDOM FOREST

Decision Trees ka ensemble — 100+ trees banake average nikalta hai. Overfitting kam hota hai, accuracy badhti hai. Most popular classifier hai Kaggle competitions mein. Feature importance bhi deta hai.

ACCURACY

Model ki performance measure karna — kitne predictions sahi hain. Accuracy = (correct predictions) / (total predictions). Class imbalance ho toh accuracy misleading ho sakti hai — precision aur recall bhi dekho.

LOGISTIC REGRESSION: probability predict karo

Logistic Regression binary classification ka foundation hai. Sigmoid function use karta hai jo kisi bhi value ko 0 aur 1 ke beech convert karta hai. Agar probability 0.5 se zyada hai toh class 1, kam hai toh class 0. Linear regression ka concept hai lekin output alag hai.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

# Height aur Weight se gender predict karo
X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0]) # 0=Female, 1=Male

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

lr = LogisticRegression()
lr.fit(X_train, y_train)

# Probability bhi dekh sakte ho
y_prob = lr.predict_proba(X_test)
y_pred = lr.predict(X_test)

print(f"Accuracy: {accuracy_score(y_test, y_pred):.2%}")
print(f"Probabilities:\n{y_prob}")

ℹ

Mental model: Logistic Regression jaise coin toss ka model hai — probability deta hai ki result kya hoga. Sigmoid curve S-shaped hai, 0 se 1 tak smoothly jaata hai. Negative values ko 0 ke paas, positive ko 1 ke paas le jaata hai.

DECISION TREE: rules se decide karo

Decision Tree human decision making jaisa hai — pehle question puchho, uske jawab se agla question decide hota hai. Tree ka har node ek condition hai (height > 170?), har leaf ek prediction hai. Interpret karna bahut asaan hai — koi black box nahi hai.

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
import numpy as np

X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

dt = DecisionTreeClassifier(max_depth=3, random_state=42)
dt.fit(X_train, y_train)

y_pred = dt.predict(X_test)
print(f"DT Accuracy: {accuracy_score(y_test, y_pred):.2%}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred))

ℹ

Pro tip: max_depth set karo overfitting se bachne ke liye. Depth zyada ho toh tree training data yaad rakh leta hai, naya data pe fail hota hai. 3-5 depth rakho generally — balance rakho simplicity aur accuracy ka.

RANDOM FOREST: 100 trees ka wisdom

Random Forest Decision Trees ka ensemble hai — 100+ trees banata hai aur unka majority vote leta hai. Har tree thoda different data pe train hota hai (bagging). Isliye overfitting kam hota hai aur generalization better hota hai. Feature importance bhi milta hai — kaunsa feature kitna important hai.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
import numpy as np

X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 100 trees ka forest
rf = RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(X_train, y_train)

y_pred = rf.predict(X_test)
print(f"RF Accuracy: {accuracy_score(y_test, y_pred):.2%}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred))

# Feature importance — kaunsa feature kitna useful
print("\nFeature Importance:")
print(f"Height: {rf.feature_importances_[0]:.3f}")
print(f"Weight: {rf.feature_importances_[1]:.3f}")

ℹ

Ensemble kyun kaam karta hai: Ek tree galat ho sakta hai, lekin 100 trees galat nahi ho sakte. Jaise 100 doctor se opinion lo — majority sahi diagnosis dega. Bootstrap aggregating (bagging) se har tree ko different data milta hai, isliye diversity aati hai.

COMPARISON: kaunsa algorithm kab use karo

Har algorithm ka apna strength hai. Logistic Regression simple aur fast hai, Decision Tree explain karna asaan hai, Random Forest accurate hai. Problem dekho, data dekho, phir choose karo.

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Sab algorithms compare karo
models = {
 'Logistic Regression': LogisticRegression(),
 'Decision Tree': DecisionTreeClassifier(random_state=42),
 'Random Forest': RandomForestClassifier(n_estimators=100, random_state=42)
}

for name, model in models.items():
 model.fit(X_train, y_train)
 y_pred = model.predict(X_test)
 acc = accuracy_score(y_test, y_pred)
 print(f"{name}: {acc:.2%}")

ACCURACY: model kaisa hai

Accuracy sabse basic metric hai — kitne predictions sahi hain. Lekin accuracy sirf tab kaam karta hai jab classes balanced ho. Agar 95% data class 0 hai toh model sabko 0 predict karega aur 95% accuracy aa jaayegi — fake accuracy. Isliye precision, recall, F1-score bhi dekho.

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
import numpy as np

X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

rf = RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)

# Har metric alag cheez batata hai
print("=== METRICS ===")
print(f"Accuracy: {accuracy_score(y_test, y_pred):.2%}")
print(f"Precision: {precision_score(y_test, y_pred, zero_division=0):.2%}")
print(f"Recall: {recall_score(y_test, y_pred, zero_division=0):.2%}")
print(f"F1 Score: {f1_score(y_test, y_pred, zero_division=0):.2%}")

print("\n=== CONFUSION MATRIX ===")
print(confusion_matrix(y_test, y_pred))

print("\n=== FULL REPORT ===")
print(classification_report(y_test, y_pred, zero_division=0))

ℹ

Metric decoder: Precision = kitne predicted positives actually positive hain. Recall = kitne actual positives ko correctly catch kiya. F1 = precision aur recall ka harmonic mean — dono ka balance. Confusion matrix se pata chalta hai kitne false positives aur false negatives hain.

Try it: classifier build karo

Editor mein sample data diya hai — Decision Tree aur Random Forest dono try karo. Naye data points bhi add karo aur predict karo. "Run Python" dabao aur dekho kaunsa better perform karta hai.

Classification playgroundAlgorithms compare karo aur accuracy check karo

main.py

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
import numpy as np

# Height (cm) aur Weight (kg) se gender predict karo
X = np.array([[170,60],[175,70],[160,55],[180,80],[165,65],[155,50],
 [172,68],[168,58],[182,85],[158,52],[177,72],[163,60]])
y = np.array([0,1,0,1,0,1,1,0,1,0,1,0]) # 0=Female, 1=Male

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Decision Tree
dt = DecisionTreeClassifier(random_state=42)
dt.fit(X_train, y_train)
dt_pred = dt.predict(X_test)
print(f"DT Accuracy: {accuracy_score(y_test, dt_pred):.2%}")

# Random Forest — 100 trees
rf = RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(X_train, y_train)
rf_pred = rf.predict(X_test)
print(f"RF Accuracy: {accuracy_score(y_test, rf_pred):.2%}")

# New prediction
new_person = np.array([[173, 62]])
print(f"\nNew person (173cm, 62kg):")
print(f" DT predicts: {'Male' if dt.predict(new_person)[0] else 'Female'}")
print(f" RF predicts: {'Male' if rf.predict(new_person)[0] else 'Female'}")

print("\n=== Classification Report (Random Forest) ===")
print(classification_report(y_test, rf_pred))

Optional input

Run Python dabayein

Quick check

Decision Tree aur Random Forest mein kya fark hai?

Socho — ek tree aur 100 trees. Kaun zyada reliable hoga✓ Random Forest kaise banaya jaata hai?

Common beginner mistakes

Accuracy pe mat jao: 95% accuracy dikha raha hai toh sochte ho model achha hai. Lekin class imbalance hai toh accuracy fake ho sakti hai. Hamesha confusion matrix aur classification report dekho.
Decision Tree mein depth control karna bhoolna: Bina max_depth ke tree poora data yaad rakh leta hai — overfitting. Training pe 100%, test pe 60%. Depth 3-5 rakho, pruning karo.
Feature scaling bhoolna: Logistic Regression ko scaling chahiye. Decision Tree aur Random Forest ko nahi chahiye. Algorithm dekh ke scaling karo, blindly mat karo.
Train test split na karna: Sab data pe train karo toh overfitting pakka hai. Hamesha 20-30% data test ke liye chhodo. Random state set karo results reproduce karne ke liye.
Random Forest mein n_estimators kam rakhna: 10 trees se kaam nahi chalta — 100+ rakho. Zyada trees = better performance, lekin speed thodi slow hoti hai. 100-500 range best hai.

Classification clear?

Ab Clustering par chalo — unsupervised learning se groups dhundho. Classification mein labels hote hain, clustering mein nahi.

CATEGORIESKI PREDICTION.