Euler's method

The simplest numerical ODE solver: follow the tangent for a small time step, $y_{k+1} = y_k + h\,f(t_k, y_k)$. First order (error $O(h)$), easy to destabilize — and, in its semi-implicit form, the default integrator of game physics.

Most ODEs have no closed-form solution. Euler's idea (1768): we know the slope at the current point, so assume it stays constant for a short time, step, and repeat. It turns any ODE into a loop.

Walking with your eyes closed, opening them every $h$ seconds to check your heading. Small $h$: you follow the path closely. Large $h$: you drift off, and on a circular orbit you spiral outwards because each tangent step leaves the circle. Error per step $O(h^2)$, accumulated over $1/h$ steps: $O(h)$.

Explicit Euler: $y_{k+1} = y_k + h\,f(t_k, y_k)$. Global error $\max_k|y(t_k) - y_k| = O(h)$ for Lipschitz $f$. On the test equation $y' = \lambda y$ ($\lambda < 0$) it is stable only if $|1 + h\lambda| \le 1$, i.e. $h \le 2/|\lambda|$.

t, y = t0, y0
while t < T:
    y = y + h * f(t, y)
    t = t + h

For mechanics, update velocity first and then position with the new velocity (semi-implicit): energy stays bounded instead of growing.

$y' = -2y$, $y(0) = 1$ (exact $e^{-2t}$), step $h = 0.1$: $y_k = 0.8^k$, so $y(1) \approx 0.8^{10} = 0.107$ vs $e^{-2} = 0.135$. With $h = 1.5$: $y_k = (-2)^k$ — the numerical solution explodes while the true one decays. That is numerical instability.

Every game engine steps its world with a variant of Euler at 60 Hz. And the most important algorithm of machine learning is Euler's method in disguise: gradient descent is explicit Euler applied to the gradient flow $\dot\theta = -\nabla L$, with the learning rate as the time step — so its stability limit $\eta < 2/\lambda_{\max}$ is Euler's stability limit.