Skip to content

Differentiation rules

Level FundamentalDifficulty ★★★★★Concept⌖ Open in the map

What is it?

The derivative is linear, and there are rules for products and quotients. With the chain rule, they let you differentiate any formula mechanically — which is why computers can do it.

Formulas

$(af + bg)' = af' + bg'$
linearity: $(fg)' = f'g + fg'$
product rule (Leibniz): $\left(\frac fg\right)' = \frac{f'g - fg'}{g^2}$
quotient rule

Where it shows up in computing

Symbolic computation (CAS)★★★★★fundamentalScientific computing and algorithms
Each rule is a rewrite rule on the expression tree; the hard part is simplifying the result.

Where it shows up in AI

Automatic differentiation★★★★★fundamentalAI and machine learning
AD applies exactly these rules, one elementary operation at a time, to numbers instead of formulas.

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

λ Scientific computing and algorithms

Symbolic computation (CAS)★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Fundamental theorem of calculus→Cumulative distribution function→Monte Carlo methods★★★★★

ℒ AI and machine learning

Automatic differentiation★★★★★
Derivatives of elementary functions→Activation functions★★★★★
Automatic differentiation→Backpropagation★★★★★
Derivatives of elementary functions→Activation functions→Neural networks★★★★★
Automatic differentiation→Backpropagation→Deep learning★★★★★
Derivatives of elementary functions→Activation functions→Neural networks→Loss landscape★★★★★
+6

⚙ Robotics and control

Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Control theory★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Control theory→Trajectory optimization and MPC★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Laplace transform→PID control★★★★★

⚛ Physics and simulation

Integration by parts→Finite element method★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Classical mechanics★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Physics engines★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Population and epidemic models★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Physics engines→N-body gravitational simulation★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Partial differential equations→Heat equation and diffusion★★★★★
+2

∿ Signals, media and vision

Derivatives of elementary functions→Antiderivatives and indefinite integrals→Partial fractions→Digital filters★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Integration by substitution→Trigonometric integrals→Signal processing★★★★★
Derivatives of elementary functions→Antiderivatives and indefinite integrals→Ordinary differential equations→Partial differential equations→Image processing and computer vision★★★★★

⇄ Optimization and systems

Derivatives of elementary functions→Antiderivatives and indefinite integrals→Fundamental theorem of calculus→Cumulative distribution function→Queueing theory and performance★★★★★

What depends on it

Exercises

1Computation

Differentiate $f(x) = \frac{x^2 e^x}{1 + x}$ .

Solution

$f'(x) = \frac{(2x + x^2)e^x(1 + x) - x^2 e^x}{(1+x)^2} = \frac{x e^x (x^2 + 2x + 2)}{(1 + x)^2}$ .

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.