Free online courseMatrix Calculus For Machine Learning

Course Description

Matrix calculus is the language that turns machine learning ideas into trainable models. When your code needs to compute gradients through vectors, matrices, decompositions, or probabilistic layers, intuition alone stops being enough—you need reliable tools for derivatives in higher dimensions. This free online course helps you build that toolkit, connecting the math directly to the workflows used in modern AI and machine learning.

You will develop a clear understanding of derivatives as linear operators and learn to express changes in matrix-valued functions in a way that stays consistent and checkable. The course bridges the gap between symbolic manipulation and practical computation by showing how Jacobians, vectorization, Kronecker products, and finite-difference approximations relate to real engineering tasks, including debugging gradients and validating numerical results.

From there, the focus expands to gradients and inner products in more general vector spaces—essential when you move beyond basic Euclidean assumptions. You will see how root finding and optimization methods connect to differentiation, and why adjoint-based techniques are central when the cost of naive gradient computation becomes too high.

A key theme is how derivatives are computed in practice. You will explore automatic differentiation concepts, dual numbers, and computational graphs, gaining insight into why reverse-mode methods underpin backpropagation and scale so well for learning problems. The course also connects differentiation to dynamical systems via adjoint differentiation of ODE solutions, offering a pathway into neural ODEs and differentiable simulation.

Later topics address derivatives of random functions, including ideas such as reparameterization for low-variance gradient estimates, and the role of second derivatives and Hessians when curvature matters for optimization and uncertainty. You will also gain perspective on differentiating eigenproblems, which appear in dimensionality reduction, spectral methods, and many advanced model components. Throughout, exercises reinforce the concepts so you can translate them into more stable implementations and stronger model understanding.