Mean (Average)
μ = (Σ xᵢ) / n
Center of a numeric set: sum of values divided by count.
Example · Values 2, 4, 6, 8 → (2+4+6+8)/4 = 5
Reference
Statistics & Algorithms
Basic statistics, advanced statistics, and common ML algorithm formulas — each with a short worked example.
Basic Statistics
Core summary measures used in every data and ML workflow.
Median
Middle value after sorting (average of two middle if even n)
Robust center that resists extreme outliers better than the mean.
Example · 1, 3, 9 → median = 3; 1, 3, 7, 9 → median = (3+7)/2 = 5
Mode
Most frequent value(s)
Typical category or repeated numeric value in a sample.
Example · Red, Blue, Red, Green → mode = Red
Variance
σ² = Σ (xᵢ − μ)² / n (population) · s² = Σ (xᵢ − x̄)² / (n−1) (sample)
Average squared distance from the mean — spread of the data.
Example · 2, 4, 6 · μ=4 → ((2−4)²+(4−4)²+(6−4)²)/3 = 8/3 ≈ 2.67
Standard Deviation
σ = √σ²
Spread in the same units as the data (square root of variance).
Example · If σ² = 2.67, then σ ≈ 1.63
Range
Range = max(x) − min(x)
Simple span from smallest to largest value.
Example · 3, 5, 12 → range = 9
Percentile / Quartile
Pₖ ≈ value below which k% of observations fall
Q1≈25th, Q2=median, Q3≈75th percentile.
Example · In exam scores, 90th percentile means scored better than ~90% of students
Correlation (Pearson)
r = Σ (xᵢ−x̄)(yᵢ−ȳ) / √[Σ(xᵢ−x̄)² Σ(yᵢ−ȳ)²]
Linear association between two variables (−1 to +1).
Example · Height vs weight often r ≈ 0.6 to 0.8 (positive relationship)
Advanced Statistics
Inference, distributions, and model-evaluation math used in ML.
Z-Score
z = (x − μ) / σ
How many standard deviations a point is from the mean.
Example · x=80, μ=70, σ=5 → z = (80−70)/5 = 2
Confidence Interval (mean)
x̄ ± z_(α/2) · (σ / √n)
Range likely containing the true mean at chosen confidence.
Example · x̄=50, σ=10, n=100, 95% → 50 ± 1.96·1 = [48.04, 51.96]
Bayes’ Theorem
P(A|B) = P(B|A)·P(A) / P(B)
Update belief about A after observing evidence B.
Example · Spam filter: update P(spam|words) from prior P(spam) and likelihood
Entropy
H(X) = − Σ p(x) log₂ p(x)
Uncertainty / impurity of a discrete distribution.
Example · Fair coin: H = −(0.5 log₂ 0.5 × 2) = 1 bit
KL Divergence
D_KL(P∥Q) = Σ P(x) log(P(x)/Q(x))
How different distribution P is from reference Q (not symmetric).
Example · Used when comparing model probability outputs to true labels
Mean Squared Error
MSE = (1/n) Σ (yᵢ − ŷᵢ)²
Average squared prediction error for regression.
Example · y=[3,5], ŷ=[2,5] → ((1)²+(0)²)/2 = 0.5
R² (Coefficient of Determination)
R² = 1 − SS_res / SS_tot
Fraction of variance explained by the model (1 is perfect fit).
Example · SS_res=20, SS_tot=100 → R² = 0.80
Precision, Recall, F1
Precision=TP/(TP+FP) · Recall=TP/(TP+FN) · F1=2·P·R/(P+R)
Classification quality when classes are imbalanced.
Example · P=0.8, R=0.6 → F1 = 2·0.8·0.6/(0.8+0.6) ≈ 0.69
ML Algorithm Formulas
Key equations behind common machine-learning algorithms, with short examples.
Linear Regression
ŷ = β₀ + β₁x₁ + … + βₚxₚ · Loss = MSE
Predict continuous target as weighted sum of features.
Example · Price ≈ 50 + 0.8·sqft → 1000 sqft predicts 850
Logistic Regression
P(y=1|x) = 1 / (1 + e^(−(β₀+βᵀx)))
Maps a linear score to a probability for binary classification.
Example · Score z=2 → P ≈ 1/(1+e^(−2)) ≈ 0.88
k-Nearest Neighbors
d(a,b) = √Σ (aᵢ−bᵢ)² · vote among k nearest
Classify/regress from majority (or average) of closest training points.
Example · k=3 neighbors: A,A,B → predict class A
Naive Bayes
P(y|x) ∝ P(y) Π P(xᵢ|y)
Assumes features independent given the class; fast probabilistic classifier.
Example · Email spam: combine prior P(spam) with word likelihoods
Decision Tree (Information Gain)
IG = H(parent) − Σ (|child|/|parent|) H(child)
Split feature that most reduces impurity/entropy.
Example · Pick Age split if it lowers class entropy more than Income
Support Vector Machine
min (1/2)||w||² + C Σ ξᵢ s.t. yᵢ(w·xᵢ+b) ≥ 1−ξᵢ
Find maximum-margin separator (with soft margin for noisy data).
Example · Two clusters → hyperplane farthest from nearest support vectors
K-Means
J = Σᵢ Σ_{x∈Cᵢ} ||x − μᵢ||² · update μᵢ = mean(Cᵢ)
Cluster points into k groups minimizing within-cluster squared distance.
Example · k=2 on customer spend/visits → two segments: casual vs heavy buyers
PCA
X = UΣVᵀ · project onto top eigenvectors of covariance
Compress correlated features into uncorrelated principal components.
Example · 100 features → keep first 10 PCs explaining 95% variance
Gradient Descent
θ ← θ − η ∇L(θ)
Iteratively update parameters opposite the loss gradient.
Example · η=0.01, ∇L=2.0 → θ decreases by 0.02 each step
Neural Net (one layer)
a = f(Wx + b) · Softmax: e^{zᵢ}/Σ e^{zⱼ}
Nonlinear transform of inputs; softmax for multi-class probabilities.
Example · z=[2,1,0] → softmax ≈ [0.66, 0.24, 0.09]
Random Forest (idea)
ŷ = majority / average of T trees trained on bootstrap samples
Ensemble of trees reduces variance vs a single deep tree.
Example · 101 trees vote: 67 Yes, 34 No → predict Yes
Gradient Boosting (idea)
Fₘ(x) = Fₘ₋₁(x) + ν · hₘ(x) where hₘ fits residuals
Add weak learners that correct previous model’s errors.
Example · XGBoost/LightGBM iteratively improve residual predictions