Mean value theorem

Somewhere on $(a, b)$ the instantaneous rate of change equals the average rate: $f'(c) = \frac{f(b) - f(a)}{b - a}$. It is the bridge from derivatives to inequalities — and therefore to every error bound in numerical analysis.

Derivatives are local; we often need global statements: "if the derivative is small everywhere, the function barely changes", "if $f' > 0$ the function increases", "this numerical method's error is at most…". The mean value theorem converts one into the other.

Draw the secant from $(a, f(a))$ to $(b, f(b))$ and slide it parallel to itself: at the last moment it touches the graph, it is a tangent. Physically: if you drove 120 km in one hour, at some instant your speedometer read exactly 120 km/h — which is how average-speed cameras can fine you.

Subtract the secant line from $f$: the difference $g$ satisfies $g(a) = g(b)$, so Rolle gives $g'(c) = 0$, i.e. $f'(c)$ equals the slope of the secant.

Whenever a numerical analyst proves that a method's error is at most $C h^p$, a mean value theorem (or its big brother, Taylor's theorem with remainder) is somewhere in the proof. Convergence proofs for gradient descent, Newton's method and ODE solvers all go through it.