Mathematics for Machine Learning and AI

Machine learning can look like magic, but underneath every model is mathematics you can learn. If you are a BS Computer Science, Data Science or AI student, four areas matter most: linear algebra, calculus, probability and optimisation. This guide explains what each one does inside a model and works through examples step by step.

Why Maths Matters in AI

Libraries such as scikit-learn and PyTorch hide the maths, so you can train a model without understanding it. The trouble comes when the model fails. Knowing the maths helps you:

• understand why a model is overfitting or not learning at all,
• choose sensible learning rates and loss functions,
• read research papers and documentation without getting lost,
• do well in university courses that test derivations, not just code.

Linear Algebra: The Language of Data

In machine learning, data lives in vectors and matrices. A single student record with height, weight and age is a vector, and a dataset of 1,000 students is a \( 1000 \times 3 \) matrix.

Key ideas to master:

• Vectors and dot products: the dot product \( \mathbf{w} \cdot \mathbf{x} = w_1x_1 + w_2x_2 + \cdots + w_nx_n \) is the basic calculation inside every neuron.
• Matrix multiplication: a whole layer of a neural network is one matrix-vector product.
• Eigenvalues and eigenvectors: these drive Principal Component Analysis (PCA), which reduces the number of features.
• Norms: the length \( \|\mathbf{x}\| = \sqrt{x_1^2 + \cdots + x_n^2} \) measures distance and is used in regularisation.

Worked Example 1: One Artificial Neuron

A neuron has weights \( \mathbf{w} = (2, -1, 0.5) \), bias \( b = 1 \), and receives input \( \mathbf{x} = (3, 4, 2) \). It uses the sigmoid activation \( \sigma(z) = \dfrac{1}{1 + e^{-z}} \). Find its output.

Step 1: Compute the dot product.

$$ \mathbf{w} \cdot \mathbf{x} = (2)(3) + (-1)(4) + (0.5)(2) = 6 - 4 + 1 = 3. $$

Step 2: Add the bias.

$$ z = 3 + 1 = 4. $$

Step 3: Apply the sigmoid. Using \( e^{-4} \approx 0.0183 \):

$$ \sigma(4) = \frac{1}{1 + 0.0183} = \frac{1}{1.0183} \approx 0.982. $$

The neuron outputs about 0.982, which a classifier could read as "98% confident this is class 1".

Calculus: How Models Learn

Training a model means changing its parameters to reduce a loss function, a number that measures how wrong the predictions are. Calculus tells us which direction to move.

• Derivatives measure how fast the loss changes when one parameter changes.
• Partial derivatives and gradients handle models with many parameters. The gradient \( \nabla L \) points in the direction of steepest increase.
• The chain rule is the heart of backpropagation, which computes gradients layer by layer through a neural network.

Worked Example 2: Gradient Descent by Hand

Minimise \( f(w) = (w - 3)^2 \) starting at \( w_0 = 0 \) with learning rate \( \eta = 0.1 \).

Step 1: Find the derivative.

$$ f'(w) = 2(w - 3). $$

Step 2: Write the update rule.

$$ w_{n+1} = w_n - \eta f'(w_n). $$

Step 3: Iterate.

• \( w_1 = 0 - 0.1 \times 2(0 - 3) = 0 - 0.1 \times (-6) = 0.6 \)
• \( w_2 = 0.6 - 0.1 \times 2(0.6 - 3) = 0.6 - 0.1 \times (-4.8) = 1.08 \)
• \( w_3 = 1.08 - 0.1 \times 2(1.08 - 3) = 1.08 - 0.1 \times (-3.84) = 1.464 \)

Step 4: Spot the pattern. Each step gives \( w_{n+1} - 3 = 0.8(w_n - 3) \), so

$$ w_n = 3 - 3(0.8)^n. $$

Check: \( n = 3 \) gives \( 3 - 3(0.512) = 3 - 1.536 = 1.464 \). As \( n \to \infty \), \( (0.8)^n \to 0 \), so \( w_n \to 3 \), the true minimum. If \( \eta \) were greater than 1, the factor \( 1 - 2\eta \) would have size greater than 1 and the steps would overshoot and diverge, which is exactly why learning rates matter.

Probability and Statistics: Handling Uncertainty

Real data is noisy, and many models output probabilities rather than certainties. You will need:

• Random variables, mean and variance,
• Common distributions: Bernoulli, binomial, normal,
• Conditional probability and Bayes' theorem, used in Naive Bayes classifiers and spam filters,
• Maximum likelihood estimation, which is how many loss functions are derived.

Worked Example 3: Bayes' Theorem in a Classifier

A disease affects 1% of people. A test detects it 95% of the time when present, and gives a false positive 5% of the time when absent. If a person tests positive, what is the probability they have the disease?

Step 1: List the probabilities. \( P(D) = 0.01 \), \( P(\text{no } D) = 0.99 \), \( P(+ \mid D) = 0.95 \), \( P(+ \mid \text{no } D) = 0.05 \).

Step 2: Find the total probability of a positive test.

$$ P(+) = (0.95)(0.01) + (0.05)(0.99) = 0.0095 + 0.0495 = 0.059. $$

Step 3: Apply Bayes' theorem.

$$ P(D \mid +) = \frac{P(+ \mid D)P(D)}{P(+)} = \frac{0.0095}{0.059} \approx 0.161. $$

Only about 16% of positives actually have the disease. This is why accuracy alone is a poor measure on imbalanced datasets, and why precision and recall exist.

Optimisation: Putting It All Together

Optimisation combines the three areas above. You define a loss, compute its gradient and update the parameters. Topics to study include convex functions (where any local minimum is global), stochastic gradient descent, momentum, and regularisation terms such as \( \lambda \|\mathbf{w}\|^2 \) that discourage overly large weights.

A Practical Learning Path

1. Weeks 1–3: vectors, matrices, matrix multiplication, determinants and eigenvalues.
2. Weeks 4–6: derivatives, partial derivatives, gradients and the chain rule.
3. Weeks 7–9: probability rules, Bayes' theorem, distributions, expectation and variance.
4. Weeks 10–12: gradient descent and linear regression from scratch in Python with NumPy.

Code each idea after you learn it. Implementing gradient descent yourself teaches more than reading about it ten times.

Key Takeaways

• Linear algebra represents data and models; a neuron is a dot product plus a bias passed through an activation.
• Calculus, especially the chain rule, is how models compute gradients and learn.
• Probability and Bayes' theorem explain uncertainty and why accuracy can mislead.
• Gradient descent repeatedly steps against the gradient, and the learning rate controls whether it converges.
• Learn each topic, then implement it in code to make it stick.

Stuck on a linear algebra derivation or a machine learning assignment? Amathify's subject experts and PhD scholars provide clear, step-by-step solutions. Contact us on WhatsApp or submit your problem through the "Ask a Question" form.