I have a simple optimization problem, but somewhere I am making a mistake. I want to maximize the entropy for the four $p_1, p_2, p_3, p_4$. I want to use Lagrange multiplier with constraints. Thus:
$\sum_i p_i \log_2(p_i)$ for $i$ of the interval $[1,4]$
s.t.
$g(p_1,p_2,p_3,p_4) = p_1+p_2+p_3+p_4 = 1$ .
Now I want to compute it with Lagrange multiplier:
$$L(p_1,p_2,p_3,p_4,\lambda)= -p_1 \log p_1 - p_2 \log p_2 - p_3 \log p_3 - p_4 \log p_4- \lambda(p_1 + p_2 + p_3 + p_4 - 1)$$
If I compute the partial derivatives:
$$\frac{\partial L}{\partial p_i} = -\log p_i - 1 - \lambda= 0$$
Thus:
$$p_i = - e - e^{\lambda}$$
If I now insert it in the Lagrange function $L$:
$$L(p_1,p_2,p_3,p_4,\lambda)= -p_1 \log p_1 - p_2 \log p_2 - p_3 \log p_3 - p_4 \log p_4- \lambda(p_1 + p_2 + p_3 + p_4 - 1)$$
I will have negative values in the log:
$$-(- e - e^{\lambda} \log(-e-e^{\lambda}))$$
which is not solvable.
I don't know where my mistake is.
Since $\frac{\partial L}{\partial p_i} = -\log p_i - 1 - \lambda = 0$, we have $$p_i = \mathrm{e}^{-\lambda - 1}.$$ As a result, $$p_1 = p_2 = p_3 = p_4 = \frac{1}{4},$$
i.e., the maximum entropy distribution should be the uniform distribution.