What is it?
Computing with qubits: states are unit vectors of complex amplitudes, gates are unitary matrices (exponentials ), measurement samples with probabilities . Shor's algorithm relies on a quantum Fourier transform. Mostly linear algebra; calculus enters through complex exponentials, Fourier analysis and the physics of the hardware.
Formulas
The mathematics behind it
Qubit amplitudes are complex numbers; interference is addition of complex phases.
Gates are , matrix exponentials defined by their power series.
Further reading
This page has the essentials. A fuller treatment (intuition, formal definition, worked example) is on the way.