Lesson 13 — Intermediate

FEATURES KAM KARO,
MODEL FAST BANAO.

Jab bahut saare features hain toh model slow hota hai aur Curse of Dimensionality aata hai. PCA se important features select karte hain — bina information loss ke dimensions reduce karte hain. Yeh technique data science ka shortcut hai jab data bahut bada ho.

WHY: Dimensionality Reduction kyun zaroori hai?

Socho tumhare paas 1000 columns hain aur sirf 100 rows. Model ko train karna hai — lekin 1000 features ke saath model confuse ho jaata hai. Yeh problem hai Curse of Dimensionality. Zyada features = zyada computation time, overfitting ka risk, aur model samajh nahi paata kaunsa feature important hai. PCA (Principal Component Analysis) yeh solve karta hai — naye features banata hai jo old features ka compressed version hai, lekin zyada information retain karta hai.

HOW: PCA with code examples

PCA step-by-step: pehle data scale karo, phir PCA fit karo, aur phir naye components lo. Scree plot se decide karo kitne components rakhne hain.

1. Basic PCA — 10 features ko 3 mein reduce karo

Pehle high-dimensional data banao, scale karo, phir PCA lagao. Print karo original shape, reduced shape, aur kitna variance capture hua.

from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
import numpy as np

# High dimensional data
X = np.random.rand(100, 10) # 100 samples, 10 features

# Scale karo — PCA ke liye scaling zaroori hai
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# PCA lagao — 3 components tak reduce karo
pca = PCA(n_components=3)
X_pca = pca.fit_transform(X_scaled)

print(f"Original: {X.shape}, Reduced: {X_pca.shape}")
# Original: (100, 10), Reduced: (100, 3)

print(f"Explained variance: {pca.explained_variance_ratio_}")
# Har component kitna variance capture karta hai

print(f"Total variance: {sum(pca.explained_variance_ratio_):.2%}")
# Total information retained

2. Scree Plot — Kitne components rakhne hain

Full PCA fit karo aur cumulative variance plot karo. Jahan curve flatten ho jaaye — wahan se components ka gain bahut kam hota hai. Yeh "elbow" point hai.

import matplotlib.pyplot as plt

# Full PCA — saare components lo
pca_full = PCA().fit(X_scaled)

# Cumulative variance plot — scree plot
plt.figure(figsize=(8, 5))
plt.plot(range(1, 11), np.cumsum(pca_full.explained_variance_ratio_), 'bo-')
plt.xlabel('Number of Components')
plt.ylabel('Cumulative Explained Variance')
plt.title('Scree Plot — Kitne components rakhne hain?')
plt.axhline(y=0.95, color='r', linestyle='--', label='95% variance')
plt.legend()
plt.grid(True)
plt.show()

# 95% variance chahiye toh kitne components?
cumsum = np.cumsum(pca_full.explained_variance_ratio_)
n_components_95 = np.argmax(cumsum >= 0.95) + 1
print(f"95% variance ke liye {n_components_95} components chahiye")

3. PCA on real dataset — Breast Cancer

Sklearn ka built-in dataset use karo. Original 30 features ko 2 mein reduce karo aur visualize karo — classes kitni achhi se separate ho rahi hain.

from sklearn.datasets import load_breast_cancer
import matplotlib.pyplot as plt

# Real dataset — 30 features
data = load_breast_cancer()
X, y = data.data, data.target
print(f"Original shape: {X.shape}") # (569, 30)

# Scale
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# PCA se 2 components banao
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)
print(f"Reduced shape: {X_pca.shape}") # (569, 2)
print(f"Variance retained: {sum(pca.explained_variance_ratio_):.2%}")

# Visualize karo — 2D plot
plt.figure(figsize=(8, 6))
for label in [0, 1]:
 mask = y == label
 plt.scatter(X_pca[mask, 0], X_pca[mask, 1], 
 label=data.target_names[label], alpha=0.7)
plt.xlabel(f'PC1 ({pca.explained_variance_ratio_[0]:.1%} variance)')
plt.ylabel(f'PC2 ({pca.explained_variance_ratio_[1]:.1%} variance)')
plt.title('Breast Cancer — PCA se 2D visualization')
plt.legend()
plt.show()

4. Choose components — Variance threshold method

Koi specific percentage nahi chahiye toh _components` mein float daalo — 0.95 matlab 95% variance retain karo. PCA automatically decide karega kitne components chahiye.

# 95% variance retain karo — PCA khud decide karega
pca_95 = PCA(n_components=0.95)
X_reduced = pca_95.fit_transform(X_scaled)

print(f"Components chosen: {pca_95.n_components_}")
# Kitne components automatically select hue
print(f"Variance retained: {sum(pca_95.explained_variance_ratio_):.2%}")
# ~95% ya usse zyada

# Individual component contribution
for i, var in enumerate(pca_95.explained_variance_ratio_):
 print(f"PC{i+1}: {var:.2%}")

Try it: PCA practice karo

Neeche ka editor Python jaisa hai. Yahan PCA code likho aur "Run Python" dabao. Screen par output dikhenge — yeh browser-based execution hai, real Python chalega.

Dimensionality Reduction tips

• Scaling zaroori hai: PCA se pehle data scale karo (StandardScaler). Bina scale kiye PCA galat results dega kyunki large-valued features dominate karenge.
• Scree plot dekho: Hamesha scree plot banao aur dekho kitne components kitna variance capture kar rahe hain. 95% threshold common hai.
• Interpretability khoyi jaati hai: PCA ke baad original feature names khatam ho jaate hain. "PC1" kya hai — yeh samajhna mushkil hota hai. Domain expertise se interpretation karo.
• PCA sirf linear relationships: PCA linear combinations hai. Agar data mein non-linear patterns hain toh t-SNE ya UMAP try karo — yeh non-linear dimensionality reduction techniques hain.

Common Dimensionality Reduction mistakes

• Scaling skip mat karo: PCA pehle scaling maangta hai. Agar features ki units alag hain (age vs salary) toh PCA galat components dega.
• Sab features mat daalo: PCA se pehle irrelevant features hatao. PCA important features compress karta hai — garbage features se garbage components banenge.
• Over-reduction mat karo: Bahut kam components rakhne se information loss hota hai. Scree plot se decide karo — 95% variance se kam mat lo.
• Test data pe fit mat karo: PCA ka scaler test data pe fit mat karo — sirf transform karo. Warna data leakage hoga.