Monotone and bounded sequences

Level UniversityDifficulty ★★★★★Theorem⌖ Open in the map

What is it?

A monotone sequence converges if and only if it is bounded. It is the cleanest way to prove convergence without knowing the limit in advance.

If $a_1 \le a_2 \le \dots$ and $a_n \le M$ for all $n$ , then $a_n \to \sup_n a_n$ .

Let $s = \sup a_n$ (it exists by completeness). Some $a_N > s - \varepsilon$ , and monotonicity keeps every later term in $(s - \varepsilon, s]$ .

Gradient descent★★★★★frequentAI and machine learning
With a small enough step the loss decreases monotonically and is bounded below, so the loss values converge.

Computing topics reachable from here, through the chain of ideas that leads to them: