Lesson 07 � Classification

NAIVE BAYES
PROBABILITY SE CLASSIFY

Spam filter, sentiment analysis, news classification � yeh sab Naive Bayes se hota hai. Yeh algorithm Bayes theorem pe based hai aur text classification mein sabse fast aur simple hai. "Naive" isliye kehta hai kyunki yeh assume karta hai ki sab features independent hain � jo practically nahi hota, lekin phir bhi kaam karta hai!

? 20 min✓ Intermediate✓ Prerequisite: SVM

WHY: Naive Bayes kyun important hai?

Real-world mein bahut saari problems aisi hoti hain jahan humein text ko classify karna hota hai � email spam hai ya nahi, review positive hai ya negative, news kis category ki hai. Naive Bayes in sab ke liye sabse fast aur efficient algorithm hai. Training bhi quick hoti hai aur prediction bhi instant. Google ka spam filter, Gmail ke auto-replies, sentiment analysis tools � sab Naive Bayes use karte hain.

BAYES THEOREM

Probability update ka formula. P(A|B) = P(B|A) * P(A) / P(B). Iska matlab: agar humein B pata hai toh A ki probability kaise update karein. Jaise: agar email mein "FREE" word hai toh spam hone ki probability kitni badhti hai. Yeh theorem Naive Bayes ka foundation hai.

NAIVE

Features ko independent assume karta hai. Matlab: email mein "FREE" hai aur "WINNER" hai � dono alag se dekhta hai, yeh nahi sochta ki dono saath mein aaye hain. Yeh assumption technically galat hai lekin practical mein kaam karta hai. Isliye naam "Naive" hai.

PROBABILITY

Har class ke liye probability calculate karta hai. Spam ki probability = 0.85, Not Spam ki = 0.15. Jo zyada probability ho woh class select hoti hai. Probability based hone se model confident bhi bata sakta hai ki kitna sure hai prediction mein.

TEXT

Text classification mein sabse zyada use hota hai. Words ko count karke probability calculate karta hai. "great" aaya toh positive ki probability badhegi, "bad" aaya toh negative ki. Spam detection mein "free", "winner", "click" jaise words spam ki probability badhate hain.

WHAT: Bayes Theorem kya hai?

Bayes Theorem probability ka sabse powerful tool hai. Yeh batata hai ki koi event kitna probable hai jab humein koi additional information mile. Example: Delhi mein barish ki probability 30% hai. Lekin agar humein pata chale ki aasman mein baadal hain toh barish ki probability badh jaati hai � yeh Bayes Theorem hai.

Bayes Theorem:


Formula:
 P(A|B) = P(B|A) � P(A) / P(B)

 P(A|B) = A ki probability jab B pata ho (Posterior)
 P(B|A) = B ki probability jab A ho (Likelihood)
 P(A) = A ki probability bina B ke (Prior)
 P(B) = B ki probability (Evidence)

Real Example � Spam Detection:

 A = Email spam hai
 B = Email mein "FREE" word hai

 Given:
 P(spam) = 0.3 (30% emails spam hote hain)
 P("FREE"|spam) = 0.8 (spam emails mein 80% mein "FREE" hota hai)
 P("FREE") = 0.2 (sab emails mein 20% mein "FREE" hota hai)

 Find: P(spam|"FREE") = ?

 P(spam|"FREE") = P("FREE"|spam) � P(spam) / P("FREE")
 = 0.8 � 0.3 / 0.2
 = 0.24 / 0.2
 = 1.2 ? 120%? 

 Wait � yeh 100% se zyada ho gaya! Isliye denominator
 P("FREE") ko aur precisely calculate karte hain:
 
 P("FREE") = P("FREE"|spam)�P(spam) + P("FREE"|not_spam)�P(not_spam)
 = 0.8�0.3 + 0.05�0.7
 = 0.24 + 0.035
 = 0.275

 P(spam|"FREE") = 0.24 / 0.275 = 0.873 = 87.3%

 Result: "FREE" word dekhne ke baad spam ki probability
 30% se badhkar 87.3% ho gayi!

WHAT: "Naive" kyun hai?

Naive Bayes ko "naive" isliye kehte hain kyunki yeh assume karta hai ki sab features ek doosre se independent hain. Yeh assumption real life mein galat hai � lekin phir bhi algorithm kaam karta hai. Yeh sabse bada paradox hai ML mein.

Independence Assumption:
?

Naive Bayes kehta hai:
 "Main har feature ko ALAG se dekhunga"

Example � Email Classification:
 Email: "FREE FREE WINNER click here"

 Naive approach:
 ✓ P("FREE"|spam) = 0.9 (independently check)
 ✓ P("WINNER"|spam) = 0.85 (independently check)
 ✓ P("click"|spam) = 0.7 (independently check)
 
 Combined: P(spam) � 0.9 � 0.85 � 0.7
 
 Reality mein:
 ? "FREE" aur "WINNER" saath mein aate hain (correlated)
 ✓ Naive Bayes yeh nahi sochta � dono ko alag treat karta hai

Kyun kaam karta hai despite "naive" assumption:
 1. Classification ke liye sirf ORDER chahiye, exact probability nahi
 2. Correlated features bhi ranking sahi dete hain
 3. Training data mein patterns automatically capture ho jaate hain
 4. Simple hone se overfitting kam hota hai

Mathematical Formula (simplified):
 P(spam|word1,word2,...) ✓ P(spam) � P(word1|spam) � P(word2|spam) � ...
 
 ✓ means "proportional to" � exact probability nahi,
 relative comparison ke liye sufficient hai

HOW: Naive Bayes ke 3 types

Data ke hisaab se teen variants hain. Continuous data ke liye Gaussian, discrete counts ke liye Multinomial, binary features ke liye Bernoulli. Har variant ka apna use case hai.

Naive Bayes Variants:
?

1. GAUSSIAN NAIVE BAYES
 ✓ Continuous features ke liye
 ✓ Har feature ka normal distribution assume karta hai
 ✓ Use case: Iris classification, medical data
 
 Example:
 Features: [height, weight, age]
 Model: P(class|features) using Gaussian distribution
 
 from sklearn.naive_bayes import GaussianNB
 model = GaussianNB()

2. MULTINOMIAL NAIVE BAYES
 ✓ Discrete counts ke liye (text classification mein best)
 ✓ Har word ki frequency count karti hai
 ✓ Use case: Spam detection, sentiment analysis
 
 Example:
 Text: "great great movie" ? [great:2, movie:1]
 Features = word counts
 
 from sklearn.naive_bayes import MultinomialNB
 model = MultinomialNB()

3. BERNOULLI NAIVE BAYES
 ✓ Binary features ke liye (present/absent)
 ✓ Sirf batata hai word hai ya nahi, kitni baar nahi
 ✓ Use case: Document classification with binary features
 
 Example:
 Text: "great movie" ? [great:1, movie:1, bad:0]
 Features = 0 ya 1 (word present hai ya nahi)
 
 from sklearn.naive_bayes import BernoulliNB
 model = BernoulliNB()

Kaunsa use karein:
 ✓ Text data (word counts) ✓ MultinomialNB
 ✓ Continuous data (numbers) ✓ GaussianNB
 ✓ Binary data (yes/no) ✓ BernoulliNB

HOW: Text Classification kaise kaam karta hai?

Text classification Naive Bayes ka sabse popular use case hai. Har word ki frequency count karke probability calculate hoti hai. Jitna zyada word kisi class mein aata hai, utni zyada probability uss class ki.

Text Classification Example:


Training Data:
 "great movie" ✓ Positive
 "great great" ✓ Positive 
 "bad movie" ✓ Negative
 "bad bad bad" ✓ Negative

Step 1: Word Counts Calculate Karo
?
 Positive class mein:
 "great": 3 baar
 "movie": 1 baar
 Total words: 4

 Negative class mein:
 "bad": 5 baar
 "movie": 1 baar
 Total words: 6

Step 2: Probability Calculate Karo
?
 P(Positive) = 2/4 = 0.5
 P(Negative) = 2/4 = 0.5

 P("great"|Positive) = 3/4 = 0.75
 P("movie"|Positive) = 1/4 = 0.25

 P("great"|Negative) = 0/6 = 0.0
 P("movie"|Negative) = 1/6 = 0.167

Step 3: New Text Classify Karo
?
 New text: "great movie"
 
 P(Positive|"great movie") ? 0.5 � 0.75 � 0.25 = 0.09375
 P(Negative|"great movie") ? 0.5 � 0.0 � 0.167 = 0.0
 
 Result: Positive (probability zyada hai)

Smoothing (Laplace):
 Agar koi word class mein nahi aaya toh probability 0 ho jaati hai
 Isliye Laplace smoothing use karte hain � har count mein 1 add karte hain
 P("great"|Negative) = (0+1)/(6+V) jahan V = vocabulary size

TRY IT: Python mein Naive Bayes

Neeche ka editor Python jaisa hai. Yahan code likho aur "Run Python" dabao. Real scikit-learn Naive Bayes use ho raha hai � text classification example hai.

Python playgroundNaive Bayes � Spam Classification

main.py

from sklearn.naive_bayes import GaussianNB, MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

# Simple classification
X = np.array([[1,2],[2,3],[3,3],[6,1],[7,2],[8,3]])
y = np.array([0,0,0,1,1,1])

print("=== Training Data ===")
for features, label in zip(X, y):
 cls = "Class A" if label == 0 else "Class B"
 print(f" {features} ? {cls}")

X_train, X_test, y_train, y_test = train_test_split(
 X, y, test_size=0.3, random_state=42
)

model = GaussianNB()
model.fit(X_train, y_train)

print(f"\n=== GaussianNB Results ===")
print(f"Accuracy: {model.score(X_test, y_test):.2%}")

# Text classification (simplified)
# "great" = positive, "bad" = negative
texts = [[2,0,1],[3,0,0],[0,2,1],[0,3,0]] # [great,bad,neutral]
labels = [1,1,0,0]

print("\n=== Text Classification ===")
print("Features: [great, bad, neutral]")
for text, label in zip(texts, labels):
 sentiment = "Positive" if label == 1 else "Negative"
 print(f" {text} ? {sentiment}")

model = MultinomialNB()
model.fit(texts, labels)

print(f"\n=== MultinomialNB Results ===")
test_cases = [[2,0,1],[0,3,0],[1,1,0]]
for test in test_cases:
 pred = model.predict([test])[0]
 sentiment = "Positive" if pred == 1 else "Negative"
 print(f" Predict {test}: {sentiment}")

Optional input

Code ko apni info se update karke run karein

Quick check

Exercise

Naive Bayes "naive" kyun hai?

Sochho � Naive Bayes ka sabse bada assumption kya hai✓ Har feature ko independently dekhta hai, doosre features se relation nahi dekhta. Is assumption ki wajah se "naive" kehte hain.

Common beginner mistakes

Continuous data ke liye MultinomialNB use karna: MultinomialNB counts ke liye hai, continuous values ke liye GaussianNB use karo. Galat variant se accuracy bahut gir jaayegi.
Feature scaling ki zaroorat samajhna: Naive Bayes ko feature scaling ki zaroorat nahi hoti. Probability-based hai, distance-based nahi. StandardScaler lagane ki zaroorat nahi hai.
Correlated features ka dhyan na rakhna: Naive Bayes assume karta hai features independent hain. Agar features bahut correlated hain (jaise "great" aur "awesome") toh model galat ho sakta hai. Feature selection important hai.
Smoothing skip karna: Agar test data mein koi word hai jo training mein nahi tha toh probability 0 ho jaayegi. Hamesha Laplace smoothing use karo (alpha parameter).
Small dataset pe trust karna: Naive Bayes ko bahut zyada data chahiye accurate results ke liye. Kam data pe probability estimates unreliable hote hain. Minimum 1000+ samples rakho.

Naive Bayes clear?

Ab KNN par chalo � K-Nearest Neighbors jo distance-based classification karta hai. Yeh instance-based learning ka example hai.

NAIVE BAYESPROBABILITY SE CLASSIFY