Partial derivatives

$\frac{\partial f}{\partial x_i}$: the derivative with respect to one variable, holding the others fixed. Each answers "how sensitive is the output to this input?" — for a neural network, to this weight.

With many inputs, "the" rate of change depends on the direction you move. The simplest directions are the axes: move one variable, freeze the rest. Partial derivatives are those axis-aligned rates, and (for nice functions) they determine the rate in every other direction.

Slice the surface $z = f(x, y)$ with the plane $y = y_0$: you get a curve, and $\partial f/\partial x$ is its slope. Slice with $x = x_0$ for $\partial f / \partial y$. On a mountain: $\partial f/\partial x$ is how steep it is walking due east, $\partial f/\partial y$ due north.

where $e_i$ is the $i$-th unit vector. If all partials exist and are continuous near $a$ ($f \in C^1$), $f$ is differentiable at $a$; if $f \in C^2$, mixed partials commute: $\partial_x\partial_y f = \partial_y\partial_x f$ (Schwarz).

Differentiate in $x_i$ treating every other variable as a constant; all single-variable rules apply.

Squared error of a linear model with two weights: $L(w, b) = (wx + b - y)^2$. Then $\partial L/\partial w = 2(wx + b - y)\,x$ and $\partial L/\partial b = 2(wx + b - y)$. The common factor $2(\hat y - y)$ is the "error signal" that backpropagation sends to every weight.

Training computes $\partial L / \partial\theta_i$ for every parameter $\theta_i$; image processing computes $\partial I/\partial x$ and $\partial I/\partial y$ to find edges; physics writes its laws (heat, waves, fluids, electromagnetism, quantum mechanics) as equations between partial derivatives.