Lesson � Intermediate

SVM:
SUPPORT VECTOR
MACHINES.

Support Vector Machines complex data ke liye best algorithm hai. Data ko higher dimension mein map karta hai aur best separating line (hyperplane) dhundhta hai. Jab linear separation na ho, kernel trick se problem solve hota hai.

WHY: SVM kyun seekhein?

SVM un situations mein kaam aata hai jab data linearly separate na ho. Ye algorithm maximum margin wala hyperplane dhundhta hai � jo dono classes ke beech sabse bada gap rakhe. Real-world mein text classification, image recognition, bioinformatics � sab mein SVM powerful hai. Random Forest se seekhne ke baad SVM ka kernel concept clearly samajh aayega.

HOW: SVM kaam kaise karta hai

SVM ek boundary (hyperplane) dhundhta hai jo dono classes ko alag kare aur margin maximize kare. Agar data 2D mein linearly separable hai, toh seedhi line mil jayegi. Agar nahi hai, toh kernel trick use hoti hai � data ko ek aur dimension mein shift karke linear separation possible banaya jata hai.

from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

# Sample data: 2 features, 2 classes
X = np.array([[1,2],[2,3],[3,3],[6,1],[7,2],[8,3]])
y = np.array([0,0,0,1,1,1])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Linear kernel
model_linear = SVC(kernel='linear')
model_linear.fit(X_train, y_train)
print(f"Linear: {model_linear.score(X_test, y_test):.2%}")

# RBF kernel (non-linear)
model_rbf = SVC(kernel='rbf')
model_rbf.fit(X_train, y_train)
print(f"RBF: {model_rbf.score(X_test, y_test):.2%}")

HYPERPLANE & MARGIN: Maximum gap dhundhna

Hyperplane wo line hai jo data ko separate karti hai. Margin hyperplane aur support vectors ke beech ka distance hai. SVM ka goal hai margin maximize karna � kyunki zyada margin ka matlab hai model better generalize karega naye data pe.

from sklearn.svm import SVC
import numpy as np

# Support vectors kya hote hain
X = np.array([[1,2],[2,3],[3,3],[6,1],[7,2],[8,3]])
y = np.array([0,0,0,1,1,1])

model = SVC(kernel='linear')
model.fit(X, y)

# Support vectors dekho
print("Support Vectors:")
print(model.support_vectors_)
print(f"\nKitne support vectors: {model.n_support_}")
print(f"Total vectors: {len(model.support_vectors_)}")

# Hyperplane parameters
print(f"\nWeight vector: {model.coef_}")
print(f"Intercept: {model.intercept_}")

KERNEL TRICK: Dimension transform

Kernel trick SVM ki sabse powerful feature hai. Jab data linearly separate na ho, kernel data ko higher dimension mein transform karta hai jahan linear separation ho sake. RBF (Radial Basis Function) kernel sabse zyada use hota hai � ye circular patterns bhi handle kar sakta hai.

from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
import numpy as np

# Non-linear data � circle pattern
np.random.seed(42)
inner = np.random.randn(20, 2) * 0.5
outer = np.random.randn(20, 2) * 2.0
X = np.vstack([inner, outer])
y = np.array([0]*20 + [1]*20)

# Compare kernels
kernels = ['linear', 'rbf', 'poly']

for k in kernels:
 model = make_pipeline(StandardScaler(), SVC(kernel=k))
 model.fit(X, y)
 score = model.score(X, y)
 print(f"{k:8s} kernel: {score:.2%}")

# RBF with gamma tuning
model_rbf = make_pipeline(
 StandardScaler(),
 SVC(kernel='rbf', gamma='scale')
)
model_rbf.fit(X, y)
print(f"\nRBF (gamma=scale): {model_rbf.score(X, y):.2%}")

Try it: Linear vs RBF kernel compare karo

Editor mein sample data hai � dono kernels se model train karo aur accuracy compare karo. Gamma aur C parameters change karke dekho model kaise affect hota hai. Non-linear data try karo aur dekho RBF kab better hai.

Common SVM mistakes

• Feature scaling na karna: SVM distance-based hai, scaling zaroori hai. Bina scaling ke features with larger values dominate karenge. Hamesha StandardScaler ya MinMaxScaler use karo.
• Default parameters pe blindly jaana: C aur gamma parameters bahut important hai. C = regularization strength, gamma = RBF kernel ka spread. GridSearchCV se best params dhundho.
• Bohot bada dataset pe SVM use karna: SVM training time O(n�) to O(n�) hai. Lakhs of rows pe SVM slow hota hai � us case mein Random Forest ya Gradient Boosting better hai.
• Kernel choice na samajhna: Har data pe RBF best nahi hota. Pehle linear try karo � agar accuracy achhi hai toh RBF ki zaroorat nahi. Simple model zyada reliable hota hai.
• Outliers ka ignore karna: SVM outliers se affect hota hai kyunki wo support vectors pe depend karta hai. Pehle data clean karo, outliers handle karo.