Lesson 04 � Classification

LOGISTIC REGRESSION
CLASSIFY KARNA

Spam hai ya nahi✓ Student pass hoga ya fail✓ Email fraud hai ya legit✓ Yeh sab classification problems hain � aur inko solve karne ka sabse pehla algorithm hai Logistic Regression. Linear Regression ka classification version jo probability deta hai ki koi cheez kisi category mein aati hai ya nahi.

? 22 min✓ Beginner✓ Prerequisite: Linear Regression

WHY: Logistic Regression kyun classification ke liye use hota hai?

Linear regression continuous values predict karta hai � jaise price, temperature. Lekin jab question ho "spam hai ya nahi" toh humein ek discrete answer chahiye � 0 ya 1. Logistic regression exactly yahi karta hai: input data leta hai aur probability deta hai ki woh data kisi ek class mein aata hai ya nahi. Agar probability 0.5 se zyada hai toh class 1, warna class 0.

SIGMOID

Logistic regression ka core function. Yeh kisi bhi number ko 0 aur 1 ke beech convert karta hai. Formula: s(z) = 1/(1 + e^(-z)). Jab z bada hota hai toh output 1 ke paas jaata hai, jab z chhota hota hai toh 0 ke paas. Yeh probability deta hai.

DECISION BOUNDARY

Woh line (ya curve) jo do classes ko alag karti hai. Decision boundary ke ek taraf class 0 hoti hai, doosri taraf class 1. Model ka goal hai best decision boundary dhundhna jo data ko sahi se separate kare. Linear logistic regression mein boundary line hoti hai.

THRESHOLD

Default threshold 0.5 hota hai. Agar sigmoid output 0.5 se zyada hai toh prediction = 1 (positive class), agar 0.5 se kam hai toh prediction = 0 (negative class). Threshold adjust karke precision ya recall ko control kar sakte ho � jaise spam detection mein threshold 0.7 rakh sakte ho taaki legitimate mails spam mein na aaye.

BINARY

Binary classification mein sirf 2 classes hoti hain � spam/not spam, pass/fail, yes/no. Logistic regression primarily binary classification ke liye hai. Multi-class problems ke liye One-vs-Rest (OVR) ya Softmax regression use hota hai � jo internally multiple binary classifiers banata hai.

WHAT: Sigmoid function kya hai?

Sigmoid function logistic regression ka dil hai. Yeh linear equation ka output leke usko 0 aur 1 ke beech compress karta hai. Linear regression mein output kisi bhi value ho sakti hai (-8 se +8), lekin classification ke liye humein 0 aur 1 chahiye � sigmoid yahi karta hai.

Sigmoid Function:


Formula: s(z) = 1 / (1 + e^(-z))

Jahan z = w1x1 + w2x2 + ... + b (linear equation)

Kya karta hai:
✓ Input koi bhi number ho (-1000 to +1000)
✓ Output hamesha 0 aur 1 ke beech hota hai
✓ Output probability deta hai

Examples:
 z = 0 ✓ s(0) = 0.50 (50% probability)
 z = 2 ✓ s(2) = 0.88 (88% probability)
 z = -2 ✓ s(-2) = 0.12 (12% probability)
 z = 10 ✓ s(10) � 1.00 (almost certain)
 z = -10 ✓ s(-10) � 0.00 (almost impossible)

Visual:
 z: -5 -3 -1 0 1 3 5
 s: .00 .05 .27 .50 .73 .95 .99

 1.0 | ***
 | ****
 0.5 | ***?threshold
 | ****
 0.0 | ****
 +--------------------------
 -5 -3 -1 0 1 3 5

Binary Classification Rule:
 s(z) >= 0.5 ✓ Class 1 (Positive)
 s(z) < 0.5 ✓ Class 0 (Negative)

WHAT: Linear Regression vs Logistic Regression

Dono ka naam similar hai lekin kaam alag hai. Linear regression predict karta hai ("price kitni hogi"), logistic regression classify karta hai ("spam hai ya nahi"). Linear regression ka output continuous hota hai, logistic regression ka output probability hota hai.

Linear Regression vs Logistic Regression:
?

Feature | Linear Regression | Logistic Regression
-----------------+-----------------------+---------------------
Purpose | Predict value | Classify category
Output | Any real number | Probability (0-1)
Equation | y = mx + b | P = s(mx + b)
Loss Function | MSE (Mean Squared) | Log Loss (Cross-Entropy)
Decision | Continuous prediction | Class label (0/1)
Use Case | Price, Temperature | Spam, Pass/Fail

Example Problem:
 "Ghar ka size dekh ke price batao" ✓ Linear Regression
 "Email ka content dekh ke spam hai kya" ✓ Logistic Regression

Kyun Linear Regression kaam nahi karta classification mein:
✓ Linear regression ka output 0 se bahar bhi ja sakta hai (-5, 150)
✓ Probability 0-1 ke beech honi chahiye � linear regression yeh guarantee nahi deta
✓ Outliers linear boundary ko bahut distort kar sakte hain
✓ Classification mein humein probability chahiye, raw value nahi

HOW: Decision Boundary kaise kaam karta hai?

Decision boundary woh line hai jo do classes ko separate karti hai. Logistic regression ek linear decision boundary seekhta hai � matlab ek straight line (2D mein) ya plane (3D mein). Data points ko do taraf divide karta hai aur prediction karta hai ki naya point kis taraf aata hai.

Decision Boundary:


2D Example: Marks vs Study Hours ✓ Pass/Fail

 Pass (1)
 ? ? ?
 | ? ? ?
 | ? ? ✓ Decision Boundary Line
 | ? ? ?
 | ? ?
 | ? ?
 +----------?
 Study Hours

Class 0 (?) = Fail | Class 1 (?) = Pass

Decision Boundary Equation:
 w1x1 + w2x2 + b = 0

 Agar w1x1 + w2x2 + b > 0 ✓ Class 1 (Pass)
 Agar w1x1 + w2x2 + b < 0 ✓ Class 0 (Fail)

Training ka goal:
✓ Weights (w1, w2) aur bias (b) aise find karna
 ki boundary data ko best separate kare
✓ Maximize karo boundary se door wale points ki distance
✓ Minimize karo misclassified points ko

HOW: Cost Function � Log Loss

Linear regression mein MSE (Mean Squared Error) use hota hai. Lekin logistic regression mein Log Loss (Binary Cross-Entropy) use hota hai. MSE use karne se cost function non-convex ho jaata hai aur gradient descent stuck ho sakta hai. Log Loss convex hota hai aur better gradients deta hai.

Log Loss (Binary Cross-Entropy):
?

Formula:
 J = -1/m * S [y*log(y) + (1-y)*log(1-y)]

 J = Cost (jitna kam utna better)
 m = Number of examples
 y = Actual label (0 ya 1)
 y = Predicted probability (0 aur 1 ke beech)

Kyun Log Loss use hota hai:
✓ Convex function hai � ek hi global minimum hai
✓ Probability output deta hai (0-1 range mein)
✓ Outlier predictions ko heavily penalize karta hai

Example:
 Actual y=1, Predicted y=0.99 ✓ Cost = -log(0.99) = 0.01 (bahut kam)
 Actual y=1, Predicted y=0.50 ✓ Cost = -log(0.50) = 0.69 (medium)
 Actual y=1, Predicted y=0.01 ✓ Cost = -log(0.01) = 4.60 (bahut zyada!)

Gradient Descent update:
 w_new = w - a * ?J/?w
 
 a = learning rate
 ?J/?w = partial derivative of cost w.r.t. weight

TRY IT: Python mein Logistic Regression

Neeche ka editor Python jaisa hai. Yahan code likho aur "Run Python" dabao. Real scikit-learn LogisticRegression use ho raha hai � student ka pass/fail predict kar rahe hain.

Python playgroundLogistic Regression � Pass/Fail Classifier

main.py

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
import numpy as np

# Pass/Fail prediction
# Features: [study_hours, attendance_percent]
X = np.array([[5,6],[7,8],[4,5],[9,10],[3,4],[8,9],[6,7]])
y = np.array([0,1,0,1,0,1,1])

print("=== Training Data ===")
for features, label in zip(X, y):
 status = "PASS" if label == 1 else "FAIL"
 print(f" Study={features[0]}h, Attend={features[1]}% ? {status}")

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(
 X, y, test_size=0.3, random_state=42
)

# Model train
model = LogisticRegression()
model.fit(X_train, y_train)

# Predict
y_pred = model.predict(X_test)
print(f"\n=== Results ===")
print(f"Accuracy: {accuracy_score(y_test, y_pred):.2%}")
print(f"\nClassification Report:")
print(classification_report(y_test, y_pred))

# New student prediction
new_student = [[6,7]]
prediction = model.predict(new_student)
prob = model.predict_proba(new_student)
print(f"New student [Study=6h, Attend=7%]:")
print(f" Prediction: {'PASS' if prediction[0] == 1 else 'FAIL'}")
print(f" Probability: {prob[0][1]:.2%} PASS")

Optional input

Code ko apni info se update karke run karein

HOW: Multi-class classification

Logistic regression binary (2 classes) ke liye hai, lekin real-world mein kabhi-kabhi 3 se zyada classes hoti hain. Scikit-learn automatically multi-class handle karta hai � ya One-vs-Rest (OVR) ya Multinomial (Softmax) strategy use karta hai.

Multi-class Strategies:


1. ONE-vs-REST (OVR):
 ✓ Har class ke liye ek binary classifier banao
 ? 3 classes (A, B, C) ? 3 classifiers:
 Classifier 1: A vs (B+C)
 Classifier 2: B vs (A+C)
 Classifier 3: C vs (A+B)
 ✓ Jo classifier sabse zyada probability de, woh class predict

2. SOFTMAX (Multinomial):
 ✓ Ek hi model sab classes ke liye probability deta hai
 ✓ Probabilities ka sum = 1.0
 ✓ Sabse zyada probability wali class select hoti hai
 ✓ Formula: P(class_k) = e^(z_k) / S e^(z_j)

Example � Flower Classification:
 Class 0: Setosa
 Class 1: Versicolor
 Class 2: Virginica

 Model output: [0.02, 0.85, 0.13]
 ✓ Prediction: Versicolor (highest probability)

Scikit-learn mein:
 model = LogisticRegression(multi_class='multinomial')
 # Ya default � scikit-learn auto-detect karta hai

Quick check

Exercise

Logistic Regression kyun classification ke liye use hota hai?

Sochho � logistic regression ka core kya hai✓ Sigmoid function jo kisi bhi value ko 0 aur 1 ke beech compress karta hai, jo probability banti hai. Linear regression se alag kyunki uska output continuous hota hai, classification ke liye nahi.

Common beginner mistakes

Linear regression ko classification mein use karna: Linear regression ka output 0-1 ke bahar ja sakta hai. Probability chahiye toh hamesha logistic regression use karo. Linear regression sirf continuous prediction ke liye hai.
Threshold value ko blindly 0.5 rakhna: Har problem mein 0.5 best threshold nahi hota. Spam detection mein threshold 0.7 rakh sakte ho taaki legitimate mails spam mein na aaye. Precision-recall curve dekh ke threshold choose karo.
Feature scaling skip karna: Logistic regression gradient descent use karta hai. Agar features bahut alag scales pe hain (age: 0-100, salary: 0-1000000) toh convergence slow hoga. StandardScaler ya MinMaxScaler use karo.
Class imbalance ignore karna: Agar dataset mein 95% emails not-spam hain aur 5% spam hain toh model sirf "not-spam" predict karke 95% accuracy de dega. Class weights ya SMOTE use karo imbalanced data ke liye.
Overfitting se bachna: Agar training accuracy bahut zyada hai (99%) lekin test accuracy kam hai (60%) toh model overfit ho raha hai. Regularization (C parameter) increase karo ya features kam karo.

Logistic Regression clear?

Ab Decision Trees par chalo � rule-based classification jo tree structure use karta hai. Yeh non-linear problems ke liye zyada flexible hai.

LOGISTIC REGRESSIONCLASSIFY KARNA