AI & ML

Regression, gradient descent, classification, and neural networks — the mathematical core of machine learning.

Supervised Learning

Linear Regression — Fit y = wx + b by minimizing squared error.
Gradient Descent — w w - f'(w) — roll downhill to a minimum.
Logistic Regression — P(y=1) = (wx + b) — squash a score into a probability.
Cross-Entropy Loss — L = - p — punish confident, wrong predictions.
Bias–Variance Tradeoff — Error = bias² + variance + noise — balance simple vs complex.

Neural Networks & Backprop — a = (w· x + b), trained by the chain rule.
K-Means Clustering — Assign to nearest centroid, move centroid to the mean, repeat.
Principal Component Analysis — Project onto the top covariance eigenvectors to compress data.
Decision Trees — Split to cut impurity — maximize information gain.

Ridge & Lasso Regularization — L2 and L1 penalties that shrink coefficients and tame overfitting.
Support Vector Machines & Kernels — Maximum-margin classifiers and the kernel trick.
Cross-Validation & Model Selection — Estimate generalization error and tune models with k-fold CV.
Naive Bayes Classifier — Bayes' rule plus conditional independence for a fast generative classifier.
Random Forests & Bagging — Average many trees over bootstrap resamples to cut variance.
Gradient Boosting — Forward stagewise additive trees fit to the loss gradient.
k-Nearest Neighbors — Classify by majority vote of the nearest training points.
ROC Curves & AUC — Threshold sweeps, TPR vs FPR, and area under the curve.
EM & Gaussian Mixture Models — Soft clustering with latent variables: responsibilities and the EM updates.
Regularization for Deep Nets (Dropout & Weight Decay) — Weight decay, dropout, and early stopping against overfitting.
Convolutional Neural Networks — Convolution, weight sharing, output sizing, and pooling for grids.
Maximum Likelihood Estimation — Pick the parameters that make the observed data most probable.