Numerical Zoo · Optimization

Himmelblau's function

Four minima, one landscape, and several ways for a starting point to make the decision for us.

Why numerical analysts keep this one around

Himmelblau's function is a two-dimensional objective with several local minima. It is smooth, easy to evaluate, easy to plot, and complicated enough to make optimization methods reveal what information they are actually using.

Gradient descent follows local slope. Simulated annealing is allowed to wander. A different starting point can send a local method to a different basin even though the objective has not changed at all.

CMNA already includes himmelblau(), which makes this a natural house animal for the optimization chapters.

R setup

Start with the actual object.

f <- function(x) himmelblau(x)

starts <- list(
  c(0, 0),
  c(-4, 4),
  c(-4, -4),
  c(4, -4)
)

vapply(starts, f, numeric(1))

By hand first

Do enough arithmetic to see the trap.

At a starting point, gradient descent sees only the local gradient. It does not receive a map showing all four basins.

A stochastic method can cross a locally unfavorable region because its acceptance rule sometimes permits a worse move.

That means the starting point, step size, temperature, and random seed are part of the numerical experiment rather than administrative details.

What to look for

Now make it earn its reputation.

Run gradient descent from several starting points with the same step size.
Run simulated annealing several times with only the random seed changed.
Compare the final objective values before deciding whether two different final coordinates represent success or failure.

Keep this distinction:

Optimization does not merely ask where the minimum is. It asks what information the search method can use to get there.

← Back to the Numerical Zoo