Types of loss function
Contents
Types of loss function
📘 Types of Loss Functions in Machine Learning
Loss functions measure how well a model’s predictions match the actual data. They guide the training process by minimizing errors.
1. ✅ Regression Loss Functions
Used when predicting continuous values (e.g., prices, temperatures).
| Loss Function | Description |
|---|---|
| MSE (Mean Squared Error) | Squares the errors. Penalizes larger errors more. |
| MAE (Mean Absolute Error) | Uses absolute differences. Less sensitive to outliers. |
| Huber Loss | Mix of MSE and MAE. More robust to outliers. |
2. ✅ Classification Loss Functions
Used when predicting categories or labels.
| Loss Function | Description |
|---|---|
| Binary Crossentropy | For binary classification (0 or 1). |
| Categorical Crossentropy | For multi-class classification (one-hot encoded labels). |
| Sparse Categorical Crossentropy | For multi-class classification (integer labels). |
3. ✅ Ranking & Specialized Loss Functions
Used for ranking tasks, reinforcement learning, or sequence models.
| Loss Function | Description |
|---|---|
| Hinge Loss | Used in SVMs. Encourages correct class margin. |
| Triplet Loss | Used for similarity learning (e.g., face recognition). |
| KL Divergence | Measures difference between two probability distributions. |
| CTC Loss | For sequence problems with unaligned input/output (e.g., speech). |
🔁 Summary
- Use MSE or MAE for regression.
- Use Crossentropy for classification.
- Use Triplet, Hinge, or CTC for more advanced tasks.