Skip to content

Functions of several variables

Level UniversityDifficulty ★★★★★Concept⌖ Open in the map

What is it?

$f : \R^n \to \R$ (or $\R^m$ ): many inputs, one (or many) outputs. With two inputs the graph is a surface over the plane; with a million inputs — the weights of a model — we reason with its level sets and its gradient.

Formulas

$f : \R^n \to \R, \qquad (x_1, \dots, x_n) \mapsto f(x_1, \dots, x_n)$; $L(\theta) = \frac1N\sum_{i=1}^{N}\ell\big(f_\theta(x_i), y_i\big), \qquad \theta \in \R^{d},\ d \sim 10^{9}$
a training loss: a function of $d$ variables

Where it shows up in computing

Image processing and computer vision★★★★★frequentSignals, media and vision
A grayscale image is a function $I(x, y)$ sampled on a grid; a colour image is $\R^2 \to \R^3$ .

Where it shows up in AI

Loss function★★★★★fundamentalAI and machine learning
A loss is a scalar function of all the model's parameters.

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

λ Scientific computing and algorithms

Multiple integrals and change of variables→Monte Carlo methods★★★★★

ℒ AI and machine learning

Loss function★★★★★
Loss function→Gradient descent★★★★★
Loss function→Logistic regression★★★★★
Partial derivatives→Backpropagation★★★★★
Loss function→Regularization★★★★★
Loss function→Support vector machines★★★★★
+13

3D Computer graphics

Surfaces and level sets→Signed distance fields and ray marching★★★★★
Partial derivatives→Gradient→Surface normals★★★★★
Surfaces and level sets→Curvature (basic differential geometry)→Mesh processing (discrete differential geometry)★★★★★
Multiple integrals and change of variables→Monte Carlo methods→The rendering equation★★★★★
Partial derivatives→Gradient→Surface normals→Lighting and shading★★★★★
Partial derivatives→Gradient→Surface normals→Ray tracing★★★★★
+1

⚙ Robotics and control

Partial derivatives→Multivariable chain rule→Jacobian matrix→Robot Jacobian (velocity kinematics)★★★★★
Partial derivatives→Multivariable chain rule→Jacobian matrix→Inverse kinematics★★★★★
Partial derivatives→Gradient→Constrained optimization→Trajectory optimization and MPC★★★★★
Partial derivatives→Multivariable chain rule→Jacobian matrix→Equilibria and stability→Control theory★★★★★
Partial derivatives→Total differential and linearization→Kalman filter★★★★★

⚛ Physics and simulation

Partial derivatives→Heat equation and diffusion★★★★★
Partial derivatives→Partial differential equations→Wave equation★★★★★
Scalar fields→Vector fields→Electromagnetism (Maxwell's equations)★★★★★
Scalar fields→Vector fields→Fluid dynamics and CFD★★★★★
Partial derivatives→Partial differential equations→Finite element method★★★★★
Scalar fields→Vector fields→Fluid dynamics and CFD→Weather and climate modelling★★★★★
+3

∿ Signals, media and vision

Partial derivatives→Image processing and computer vision★★★★★
Partial derivatives→Image processing and computer vision→Media compression (JPEG, MP3, video)★★★★★

⇄ Optimization and systems

Loss function→Gradient descent→Recommender systems★★★★★
Partial derivatives→Gradient→Constrained optimization→Operations research and logistics★★★★★

ψ Quantum computing and physics

Partial derivatives→Divergence→Laplacian→Schrödinger equation★★★★★

What depends on it

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.