Scientific computing

What is it?

Formulas

$$\text{error} = \underbrace{O(h^p)}_{\text{truncation}} + \underbrace{O(\varepsilon_{\text{mach}}/h^q)}_{\text{rounding}}$$

Why does it matter?

The mathematics behind it

• Absolute and relative error★★★★★fundamental

  Every numerical result should come with an error estimate; truncation and rounding errors are budgeted separately.

• Conditioning★★★★★fundamental

  Ill-conditioned systems (large $\kappa$) need reformulation or higher precision, not a better solver.

• Order of convergence★★★★★fundamental

  The order of a method decides how many iterations (and how much compute) an accurate answer costs.

• Newton's method★★★★★fundamental

  The default solver for nonlinear equations and systems, usually with safeguards.

• Numerical integration (quadrature)★★★★★fundamental

  Adaptive quadrature (e.g. scipy.integrate.quad) is a standard tool of scientific software.

• Taylor's theorem and the remainder★★★★★fundamental

  Truncation errors of finite differences and ODE integrators are Taylor remainders.

• Runge–Kutta methods★★★★★fundamental

  Adaptive Runge–Kutta pairs are the standard general-purpose ODE solvers.

• Sequences★★★★★frequent

  Iterative solvers produce a sequence of approximations $x_k$ that should converge to the answer.

• Convergence of sequences★★★★★frequent

  Stopping criteria such as $|x_{k+1} - x_k| < \text{tol}$ are a finite check of the Cauchy condition.

• Bisection method★★★★★frequent

  Robust solvers (Brent's method) combine bisection's guarantee with faster steps.

• Asymptotic notation (O, o, Ω, Θ)★★★★★frequent

  Discretization errors are stated as $O(h^p)$: halve the step, divide the error by $2^p$.

• Differential and linear approximation★★★★★frequent

  First-order error propagation estimates how measurement and rounding errors affect results.

• Mean value theorem★★★★★frequent

  Error bounds of numerical methods are derived from the MVT and Taylor's theorem.

• Fixed-point iteration★★★★★frequent

  Jacobi and Gauss–Seidel solvers, and many self-consistent schemes, are fixed-point iterations.

• Taylor and Maclaurin series★★★★★frequent

  Special functions (erf, Bessel) are evaluated with series near 0 and asymptotic expansions far away.

• Stiffness and implicit methods★★★★★frequent

  Chemical kinetics and circuit simulation (SPICE) rely on stiff solvers (BDF, Rosenbrock).

• Numerical series★★★★★frequent

  Summing many terms accurately requires compensated (Kahan) or pairwise summation.

• Intervals★★★★★advanced

  Interval arithmetic computes with enclosures $[\underline x, \overline x]$ to get guaranteed error bounds.