Numerical Zoo · Floating point

The cancellation-prone quadratic

The algebra is correct. One subtraction is still a terrible idea.

Why numerical analysts keep this one around

The ordinary quadratic formula contains a subtraction between quantities that can be nearly equal. In exact arithmetic that is fine. In floating-point arithmetic, subtracting two nearly equal large values can erase the digits we hoped would distinguish them.

CMNA includes quadratic() and quadratic2() precisely because the two functions compute mathematically equivalent formulas with different numerical behavior. This is a clean example of a recurring theme in the book: an implementation is not merely a transcription of algebra.

Nothing here requires an exotic function or a large data set. A tiny quadratic is enough to make numerical representation part of the answer.

R setup

Start with the actual object.

# One root is large and the other is tiny.
b2 <- 1
b1 <- 1e8
b0 <- 1

quadratic(b2, b1, b0)
quadratic2(b2, b1, b0)

By hand first

Do enough arithmetic to see the trap.

The discriminant is b1^2 - 4*b2*b0, which is extremely close to b1^2 when b1 is large.

For one root, the ordinary formula subtracts sqrt(discriminant) from a number of almost the same size.

The alternative form uses the product relationship among the roots to avoid asking floating point to preserve a tiny remainder after that subtraction.

What to look for

Now make it earn its reputation.

Increase b1 by powers of ten and compare the small root from both implementations.
Substitute each computed root back into the original polynomial. Residuals make the damage easier to see.
This is a good example to pair with finite differences, where a different subtraction eventually creates the same class of problem.

Keep this distinction:

Mathematical equivalence does not imply numerical equivalence. That sentence is worth earning more than once.

← Back to the Numerical Zoo