Calculus×Computing
ES

Numerical differentiation

Level UniversityDifficulty ★★★★★Method⌖ Open in the map

What is it?

Estimating derivatives from function values: f(x+h)−f(x−h)2h\frac{f(x + h) - f(x - h)}{2h} has error O(h2)O(h^2). Too large an hh gives truncation error, too small an hh rounding error; the best hh for central differences in double precision is around 10−510^{-5}.

Formulas

f′(x)=f(x+h)−f(x−h)2h+O(h2)f'(x) = \frac{f(x + h) - f(x - h)}{2h} + O(h^2)
f′′(x)≈f(x+h)−2f(x)+f(x−h)h2f''(x) \approx \frac{f(x + h) - 2f(x) + f(x - h)}{h^2}
the stencil behind the discrete Laplacian

Where it shows up in computing

  • Heat equation and diffusion★★★★★fundamentalPhysics and simulation

    Finite-difference solvers replace ∂2/∂x2\partial^2/\partial x^2 by the [1,−2,1]/h2[1, -2, 1]/h^2 stencil on a grid.

  • Image processing and computer vision★★★★★frequentSignals, media and vision

    Sobel and Prewitt filters are smoothed finite differences of pixel intensities.

  • Fluid dynamics and CFD★★★★★frequentPhysics and simulation

    Grid-based fluid solvers discretize derivatives with finite differences on staggered grids.

Where it shows up in AI

  • Automatic differentiation★★★★★frequentAI and machine learning

    "Gradient checking" compares autodiff gradients against finite differences to catch bugs.

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

⚛ Physics and simulation

∿ Signals, media and vision

ℒ AI and machine learning

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.

↑ ↓ to navigate · ↵ · Esc