Hyperbolic functions

Level UniversityDifficulty ★★★★★Concept⌖ Open in the map

What is it?

$\sinh x = \frac{e^x - e^{-x}}{2}$ , $\cosh x = \frac{e^x + e^{-x}}{2}$ , $\tanh = \sinh/\cosh$ . They play for the hyperbola $x^2 - y^2 = 1$ the role $\sin$ and $\cos$ play for the circle; $\tanh$ is a classic activation function.

Formulas

\cosh^2 x - \sinh^2 x = 1, \qquad \tanh' x = 1 - \tanh^2 x

Where it shows up in AI

Activation functions★★★★★frequentAI and machine learning
$\tanh$ squashes to $(-1,1)$ and its derivative $1 - \tanh^2$ is cheap to compute from the output.

Where is it used?

Computing topics reachable from here, through the chain of ideas that leads to them:

ℒ AI and machine learning

Activation functions★★★★★
Activation functions→Neural networks★★★★★
Activation functions→Neural networks→Backpropagation★★★★★
Activation functions→Neural networks→Loss landscape★★★★★
Activation functions→Neural networks→Backpropagation→Deep learning★★★★★
Activation functions→Neural networks→Loss landscape→Second-order (Hessian-based) optimization★★★★★
+3

This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.