Skip to content

First-order linear equations

Level UniversityDifficulty ★★★★★Method⌖ Open in the map

What is it?

$y' + p(t)\,y = q(t)$ , solved with an integrating factor $e^{\int p}$ . The response of every first-order system (thermometer, RC filter, simple capacitor charging) is an exponential approach to equilibrium.

Formulas

y' + a\,y = b \implies y(t) = \frac ba + \Big(y_0 - \frac ba\Big)e^{-at}

Where it shows up in computing

Control theory★★★★★frequentRobotics and control
First-order lags with time constant $\tau = 1/a$ are the building blocks of system models.
Digital filters★★★★★frequentSignals, media and vision
The exponential moving average $y_k = (1-\alpha)y_{k-1} + \alpha x_k$ is a discretized first-order linear ODE (an RC low-pass).

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

⚙ Robotics and control

Systems of ODEs→Dynamical systems→Control theory★★★★★
Systems of ODEs→Dynamical systems→Control theory→Trajectory optimization and MPC★★★★★
Second-order linear equations (oscillations)→PID control★★★★★
Systems of ODEs→Robot dynamics★★★★★

⚛ Physics and simulation

Second-order linear equations (oscillations)→Classical mechanics★★★★★
Systems of ODEs→Population and epidemic models★★★★★
Systems of ODEs→N-body gravitational simulation★★★★★
Second-order linear equations (oscillations)→Classical mechanics→Physics engines★★★★★
Systems of ODEs→Dynamical systems→Weather and climate modelling★★★★★
Second-order linear equations (oscillations)→Classical mechanics→Physics engines→Fluid dynamics and CFD★★★★★

λ Scientific computing and algorithms

Systems of ODEs→Dynamical systems→Equilibria and stability→Stiffness and implicit methods→Scientific computing★★★★★

ℒ AI and machine learning

Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent★★★★★
Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent→Learning rate★★★★★
Systems of ODEs→Dynamical systems→Phase space→Attractors→Neural networks★★★★★
Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent→Backpropagation★★★★★
Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent→Stochastic gradient descent (SGD)★★★★★
Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent→Loss landscape★★★★★
+7

⇄ Optimization and systems

Systems of ODEs→Dynamical systems→Equilibria and stability→Gradient descent→Recommender systems★★★★★

∿ Signals, media and vision

Digital filters★★★★★

What depends on it

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.