The function $-\log x$ is a convex function, but its minimum only exists at infinity. How is this possible?

Question

Consider the function $f$ below:

$f(x) = -\log x$

Its second derivative is positive for all $x$, therefore it is convex:

$\Large \frac{d^2f}{dx^2} =\frac{d(\frac{-1}{x})}{dx} = \frac{1}{x^2}>0, \forall x$

However, setting its first derivative to zero, we find that the minimum of the function is attained at infinity? So can we still say that it has a minimum? The negative log is a very popular "loss function" in machine learning - it is interesting that they are trying to minimize a function whose minimum is literally unattainable.

Answer 1

It seems that you have misunderstood the following principle:

If $f(x)$ is a convex function, and $f'(x_0)=0$ for some $x_0$, then $f$ attains its minimum value at $x_0$.

The reason it doesn't apply here is that $f'(x)$ is never equal to $0$. So it has no minimum value.

Answer 2

There is no minimum for the function $f(x)=-\log{x}$ defined on the positive reals for precisely the reason you point out. For any $x_0>0$, we can find $y_0>x_0>0$ such that $-\log{y_0}<-\log{x_0}$. So there is no global minimum. Moreover, the first derivative is negative everywhere, so there are no local extrema either. All we can say about $f$ is that it is strictly decreasing on the interval $(0,\infty)$.

Answer 3

The $-\log$ function has no minimum or maximum as you said, but there is nothing wrong with that. In the context of machine learning we're interesting in another function, say $f$, but finding the minimum of $-\log (f(x))$ gives you the maximum of $f$, because the logarithm is monotonic. It just turns out that for certain applications it's more convenient to work with $-\log (f(x))$ instead of $f$.

Answer 4

Any function $\phi$ which is differentiable on its domain has its extrema only at points $(x,\phi(x))$ such that $\phi'(x)=0$. Since $-\log$ is differentiable on its domain $\Bbb{R}^+$, to find potential extrema we have to solve the equation $$ -\frac{1}{x}=0 \, . $$ This equation has no solution over the real numbers, and so in standard contexts we would simply say that $-\log$ has no extrema. However, if we are willing to extend the domain of $-\log$ to $\Bbb{R}^+\cup\{\infty\}$ (where "$\infty$" is an affinely extended real number), and define $-\log(\infty)=-\infty$, then $-\log$ has a minimum at $\infty$.

Answer 5

There are many functions with similar behaviour, one of elementary ones is $x\mapsto\frac 1x$. However, what you wrote does not imply the function 'has a minimum in infinity' because the function doesn't have a value in infinity, at best it has a limit there.