ML Formulas Cheatsheet 🧮
Key machine learning formulas, evaluation metrics, and concepts you must know for interviews.
📊 Evaluation Metrics
Precision = TP / (TP + FP)
"Of all predicted positive, how many are actually positive?"
Recall = TP / (TP + FN)
"Of all actual positive, how many did we catch?"
F1 Score = 2 × (Precision × Recall) / (Precision + Recall)
"Harmonic mean — balances precision and recall"
AUC-ROC: Area under TPR vs FPR curve
"Model's ability to distinguish classes across all thresholds"
R² = 1 - (SS_res / SS_tot)
"Proportion of variance explained by the model (regression)"
📉 Loss Functions
MSE = (1/n) Σ(yᵢ - ŷᵢ)²
Regression. Penalizes large errors more.
Binary Cross-Entropy = -[y·log(p) + (1-y)·log(1-p)]
Binary classification. Measures distance between predicted probability and label.
Categorical Cross-Entropy = -Σ yᵢ·log(pᵢ)
Multi-class classification.
Hinge Loss = max(0, 1 - y·ŷ)
SVM classification. Margin-based.
🧠 Key Concepts
Bias-Variance: Total Error = Bias² + Variance + Noise
Underfitting (high bias) vs Overfitting (high variance)
Gradient Descent: θ = θ - α · ∇L(θ)
α = learning rate, ∇L = gradient of loss w.r.t parameters
Attention: Attention(Q,K,V) = softmax(QKᵀ/√dₖ)V
Scaled dot-product attention (Transformers)
Softmax: σ(zᵢ) = eᶻⁱ / Σeᶻʲ
Converts logits to probabilities (sum = 1)
Bayes: P(A|B) = P(B|A)·P(A) / P(B)
Posterior = (Likelihood × Prior) / Evidence
🛡️ Regularization
L1 (Lasso): Loss + λΣ|wᵢ| → Sparse (feature selection)
L2 (Ridge): Loss + λΣwᵢ² → Small weights (no zeros)
Dropout: Randomly zero out p% of neurons during training
Batch Norm: Normalize activations to μ=0, σ=1 per mini-batch