Conditioning

Level UniversityDifficulty ★★★★★Concept⌖ Open in the map

What is it?

How much a problem amplifies relative errors in its data, independently of the algorithm. Its condition number $\kappa$ says you can lose about $\log_{10}\kappa$ digits however cleverly you compute.

Formulas

\kappa_f(x) = \left|\frac{x\,f'(x)}{f(x)}\right|, \qquad \kappa(A) = \norm A\,\norm{A^{-1}}

Where it shows up in computing

Scientific computing★★★★★fundamentalScientific computing and algorithms
Ill-conditioned systems (large $\kappa$ ) need reformulation or higher precision, not a better solver.

Where it shows up in AI

Loss landscape★★★★★frequentAI and machine learning
A Hessian with large condition number makes gradient descent zigzag; preconditioning (Adam, normalization layers) helps.

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

What depends on it

Numerical stability

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.

What is it?

Formulas

Where it shows up in computing

Where it shows up in AI

Where is it used?

λ Scientific computing and algorithms

⚛ Physics and simulation

ℒ AI and machine learning

∿ Signals, media and vision

⇄ Optimization and systems

What depends on it