Are minimizing a function and root finding the same?

Question

What is the relationship between minimizing a function and finding a root of an equation? Are the the same? I know in both problem we have similar algorithms, such as gradient decent, or newton's methods.

For example, Let use assume $x$ is a scalar. Finding a root for an equation $f(x)=b$ is checking where the function $f(x)-b$ cross the x axis. This is definitely not equal to minimize $f(x)-b$.

But in the convex optimization book, Minimizing $\|Ax-b\|_2^2$ is equal to the solution of the linear system $Ax=b$.

What I am missing? in which case we can transfer optimization to root finding?

Answer 1

Definitely not same. In your case, assuming that columns of $A$ span vector $b$ and $||Ax-b||$ being norm which is non-negative, the problem of investigating minima is similar to finding solution of $Ax-b=0$.

Answer 2

Footnote to @Roby's answer: As a practical note, if you're using an off-the-shelf optimizer to solve $f(x)=b$ by minimizing $||f(x)-b||^2_2$, you have to be careful that a minimum you find is actually a solution. As an example, suppose you're trying to solve $f(x)=b$ for $x \in \mathbb{R}$, where

$ \begin{split} f(x) &= x^2 + 2\\ b &= 1 \end{split}$

Then $x = 0$ minimizes $||f(x)-b||^2_2$, but it doesn't solve $f(x)=b$, since no real $x$ solves $x^2 + 2 = 1$.

So, to the question, you can use an optimization method to find roots, by minimizing $||f(x)-b||^2_2$, and checking that the objective-function value is $0$ at the minimum. That said, not all minima, even global minima, correspond to roots, since not all equations have roots.