Lesson 07 � Advanced

ENSEMBLE METHODS
SAB MILKE STRONG.

Ensemble multiple models ko combine karta hai � ek model se better. Kaggle competitions mein hamesha ensemble use hota hai. Jab sab models milkar decide karein toh result zyada reliable hota hai.

WHY: Ensemble Methods kyun?

Ek model ka perspective limited hota hai. Agar aap 10 alag-alag models lo aur unka average ya majority vote lo, toh result zyada reliable hota hai. Yahi ensemble ka power hai.

HOW: Ensemble Methods kaise kaam karte hain?

Ensemble learning ek meta-technique hai jo multiple models ko combine karke better performance achieve karti hai.

Types of Ensemble Methods:

1. Voting: Sab models ka prediction lo aur majority/average decide karo
2. Bagging: Random sampling se multiple models parallel mein train karo
3. Boosting: Sequentially models train karo, har ek previous ke errors fix kare
4. Stacking: Multiple models ke predictions ko meta-model ke input mein do

Key Benefits:

• Reduced Overfitting: Multiple models se noise kam hota hai
• Better Accuracy: Ek model se better performance milta hai
• Robustness: Ek model fail ho jaye toh dusre sambhal lete hain

Code Example

from sklearn.ensemble import (
 VotingClassifier, BaggingClassifier,
 AdaBoostClassifier, GradientBoostingClassifier
)
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification
import numpy as np

X, y = make_classification(n_samples=200, n_features=5, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
 X, y, test_size=0.3, random_state=42
)

# Voting � multiple models ka ensemble
voting = VotingClassifier(estimators=[
 ('dt', DecisionTreeClassifier()),
 ('rf', RandomForestClassifier()),
 ('gb', GradientBoostingClassifier())
], voting='soft')
voting.fit(X_train, y_train)
print(f"Voting: {voting.score(X_test, y_test):.2%}")

# Bagging � parallel training
bag = BaggingClassifier(n_estimators=10, random_state=42)
bag.fit(X_train, y_train)
print(f"Bagging: {bag.score(X_test, y_test):.2%}")

# Boosting � sequential improvement
boost = AdaBoostClassifier(n_estimators=50, random_state=42)
boost.fit(X_train, y_train)
print(f"Boosting: {boost.score(X_test, y_test):.2%}")

Try it: code ko run karo

Common mistakes

• Sirf ek type use karna: Alag scenarios mein alag ensemble methods better kaam karte hain � try karo kaunsa tumhare data ke liye best hai.
• Bohot zyada models: Jitne zyada models, utna zyada computation time. Balance rakho accuracy aur speed ke beech.
• Baseline bhoolna: Pehle single model ka performance check karo � tabhi pata chalega ensemble kitna improve kar raha hai.