Functions

A rule $f$ that assigns to each input $x$ of a set $A$ exactly one output $f(x)$ in a set $B$. Calculus studies how outputs change when inputs change; computing is, quite literally, evaluating functions.

To talk about dependence: the position of a planet depends on time, the cost of an algorithm on the input size, the error of a neural network on its weights. A function packages that dependence as a single object we can add, compose, differentiate and integrate.

Think of a machine: put $x$ in, get $f(x)$ out, always the same output for the same input. Its graph $\{(x, f(x))\}$ is a curve in the plane that every vertical line crosses at most once. Most of calculus is reading geometric properties of that curve — slope, area, curvature — from the formula.

A function $f : A \to B$ is a subset $G \subset A \times B$ such that for each $x \in A$ there is exactly one $y \in B$ with $(x, y) \in G$; we write $y = f(x)$. $A$ is the domain, $B$ the codomain, and $f(A) = \{f(x) : x \in A\}$ the range (image).

$f(x) = \sqrt{x - 1}$ has domain $[1, \infty)$ and range $[0, \infty)$. The softmax function of machine learning, $\operatorname{softmax}(z)_i = e^{z_i} / \sum_j e^{z_j}$, is a function $\R^n \to \R^n$ whose range is the set of probability vectors with positive entries.

A trained neural network is a function — millions of parameters fixing one particular $f : \R^n \to \R^m$. Pure functions (same input, same output, no side effects) are also the unit of reasoning in functional programming, and the reason caching and parallelism are safe.