Calculus×Computing
ES

Derivatives

The instantaneous rate of change: slope, velocity, sensitivity. The single most used idea of calculus in computing.

8 topics

If you keep one idea from calculus, keep this one. A derivative answers "if I nudge the input, how much does the output move?" — and that question is everywhere in computing:

DERIVATIVE
   ├──► Optimization ──► Gradient descent ──► Machine learning ──► Deep learning
   ├──► Physics ──► Simulation (velocity, acceleration, forces)
   ├──► Graphics ──► Normals ──► Lighting
   ├──► Robotics ──► Motion (velocity kinematics, Jacobians)
   └──► Control ──► Dynamical systems (the D in PID)

Real examples: every training step of a neural network computes millions of partial derivatives with the chain rule; a game engine integrates velocity (a derivative of position) sixty times a second; a renderer computes surface normals as derivatives of the surface; an edge detector looks for large derivatives of image intensity.

Topics

Derivative

f′(a)f'(a) is the instantaneous rate of change of ff at aa: the slope of the tangent line to the graph, defined as the limit of slopes of secant lines.

Fundamental◐ demo

Differentiability and one-sided derivatives

A function is differentiable at aa when both one-sided derivatives exist and agree. Differentiable implies continuous, but not conversely: ∣x∣|x| and ReLU⁡\operatorname{ReLU} are continuous with a kink at 0.

Fundamental

Differentiation rules

The derivative is linear, and there are rules for products and quotients. With the chain rule, they let you differentiate any formula mechanically — which is why computers can do it.

Fundamental

Derivatives of elementary functions

The table every differentiation engine starts from: powers, exponentials, logarithms, trigonometric and hyperbolic functions — and the activation functions built from them.

Fundamental

Chain rule

The derivative of a composition is the product of the derivatives: (g∘f)′(x)=g′(f(x)) f′(x)(g\circ f)'(x) = g'(f(x))\,f'(x). Rates of change multiply along a chain — and backpropagation is this rule applied, very efficiently, to a neural network.

Fundamental

Implicit differentiation

Differentiating a relation F(x,y)=0F(x, y) = 0 without solving for yy: apply the chain rule to both sides and solve for y′y'. The slope of a circle, an ellipse or an elliptic curve at a point comes out this way.

University

Higher-order derivatives

f′′f'' is the rate of change of the rate of change: acceleration, curvature, concavity. Third derivative: jerk. Robots and cameras are tuned to keep these smooth.

Fundamental

Differential and linear approximation

Near aa, f(a+h)≈f(a)+f′(a) hf(a + h) \approx f(a) + f'(a)\,h. The differential df=f′(x) dx\dd f = f'(x)\,\dd x is that best linear approximation; it tells how an error in the input propagates to the output.

University

Where this area leads in computing

λ Scientific computing and algorithms ★★★★★

ℒ AI and machine learning ★★★★★

⚙ Robotics and control ★★★★★

⚛ Physics and simulation ★★★★★

3D Computer graphics ★★★★★

∿ Signals, media and vision ★★★★★

⊕ Cryptography and security ★★★★★

↑ ↓ to navigate · ↵ · Esc