Prove the lower bound of the function $f(x) = \sum_{i = 1}^{m} x_i\log{x_i}$ subject to $\sum_{i=1}^{m} x_i =1$

Question

Can anyone prove the lower bound of the function $f(x) = \sum_{i = 1}^{m} x_i\log{x_i}$ subject to $\sum_{i=1}^{m} x_i =1$, thanks!

The lower bound is given as $f(x)\ge-\log{m}$, but why?

Answer 1

Since $\log$ is concave, Jensen's inequality yields

$$\sum_{i=1}^m x_i \log\left({\frac{1}{x_i}}\right) ≤ \log\left(\sum_{i=1}^m x_i \cdot \frac{1}{x_i}\right) = \log m$$

This inequality is sharp because equality is attained when $x_1 = \cdots = x_m = 1/m$.

Answer 2

Using Lagrange multipliers, consider $$F= \sum_{i = 1}^{m} x_i\log(x_i)+\lambda\sum_{i = 1}^{m} x_i$$ $$\frac{\partial F}{\partial x_i}=1+\lambda+\log(x_i)=0$$ So, all $x_i$ are equal and then $x_i=\frac 1m$

Answer 3

Let $x_i>0$, then $$f(x)=x \ln x \implies f''(x)=1/x>0$$ so according to Jensens inequality $$\frac{\sum_{i=1}^{n} x_i \ln x_i}{n} \ge \bar x \log \bar x=-\frac{\ln n}{n}$$ $$ \bar x=\frac{1}{n}\sum_{i=1}^{n} x_i=\frac{1}{n},~\text{given}~\sum_{i=1}^{n} x_i=1$$. Finally, $$\sum_{i=1}^{n} x_i \ln x_i \ge -\ln n$$

Answer 4

Suppose $i\ne j$ and $x_i>x_j.$ For $y\in [0,x_i-x_j]$ let $g(y)=(x_i-y)\ln (x_i-y)+(x_j+y)\ln (x_j+y).$ Then $g'(0)<0.$ So for some (sufficiently small) $y_1\in (0,x_i-x_j)$ we have $g(y_1)<g(0).$

So, in the expression for $f,$ if we replace $x_i,x_j$ with $x_i-y_1,x_j+y_1,$ we get a lower value for $f$. So $f$ is not minimized if some $x_i>x_j.$

So (i)$\,\min f$ exists,and (ii) $\,\min f$ does not occur if any two $x_i,x_j$ are unequal. Therefore $\min f$ occurs when and only when every $x_i=1/m.$