Convergence of sequences

$a_n \to L$ means: however small a tolerance $\varepsilon$ you pick, from some index on every term is within $\varepsilon$ of $L$. A sequence that does not converge diverges — to infinity, or by oscillating forever.

"Getting closer and closer" is not enough ($1/n$ gets closer and closer to $-1$ too). We need a definition that a computer could check in principle: give me any tolerance, I give you an index after which the error stays below it. That definition is also exactly what a stopping criterion in numerical software tries to certify.

Draw a horizontal band of half-width $\varepsilon$ around $L$. Convergence says that, whatever the width of the band, the dots $(n, a_n)$ eventually enter it and never leave. Narrow the band and you may have to wait longer — that waiting time $N(\varepsilon)$ is what numerical analysts call the rate of convergence.

A sequence is Cauchy if $\forall \varepsilon > 0\ \exists N\ \forall m, n \ge N: |a_m - a_n| < \varepsilon$. In $\R$ (and only because $\R$ is complete) Cauchy sequences are exactly the convergent ones.

$a_n = \frac{n}{n+1} \to 1$: given $\varepsilon$, $|a_n - 1| = \frac{1}{n+1} < \varepsilon$ as soon as $n > \frac1\varepsilon - 1$. For $\varepsilon = 10^{-6}$ you need $N = 10^6$ — slow, "sublinear" convergence. Compare with Newton's iteration for $\sqrt 2$, which reaches $10^{-15}$ in five steps.

Every iterative algorithm — training a model, solving a linear system, computing PageRank — is a sequence that we hope converges. Proving that it does, and how fast, is how we know it is worth running; and in practice the Cauchy criterion ("stop when consecutive iterates barely change") is the most common stopping rule.