Support Vector Machines
What
Find the hyperplane that best separates classes with the maximum margin — largest gap between the closest points of each class.
Key ideas
- Support vectors: the data points closest to the boundary — they define it
- Margin: distance between the boundary and the nearest points
- Kernel trick: project data into higher dimensions where it becomes linearly separable
Kernels
| Kernel | When |
|---|---|
| Linear | Data is linearly separable |
| RBF (Gaussian) | Default, works for most nonlinear cases |
| Polynomial | Specific polynomial relationships |
from sklearn.svm import SVC
model = SVC(kernel="rbf", C=1.0, gamma="scale")
model.fit(X_train, y_train)When to use
- Small to medium datasets with clear margin of separation
- High-dimensional data (works well even when features > samples)
- When you need a non-probabilistic classifier
When NOT to use
- Large datasets (training is O(n²) to O(n³))
- When you need probability outputs (use
SVC(probability=True)but it’s slow) - When interpretability matters (SVM is a black box)
Links
- Logistic Regression — linear alternative
- Feature Scaling — SVM requires scaled features
- Evaluation Metrics