I had the following exercise:
I want to maximize the entropy for the four $p_1, p_2, p_3, p_4$.
where the following condition is given:
$\sum2p_i \cdot i=4$
So my constrained optimization problem is:
$\max_{} f(p_1,p_2,p_3,p_4) = -p_1 \log p_1 - p_2 \log p_2 - \log p_3 - p_4 \log p_4$
s.t.
$ g_1(p_1,p_2,p_3,p_4) = 2p_1 + 4p_2 + 6p_3 + 8p_4 = 4$
$ g_2(p_1,p_2,p_3,p_4) = p_1 + p_2 + p_3 + p_4 = 1$
Now the lagrangian is:
$L(p_1,p_2,p_3,p_4,\lambda_1,\lambda_2) = -p_1 \log p_1 - p_2 \log p_2 - p_3 \log p_3 - p_4 \log p_4- \\\lambda_1( 2p_1 + 4p_2 + 6p_3 + 8p_4 - 4) - \lambda_2(p_1 + p_2 + p_3 + p_4 - 1)$
Then I have computed the derivatives:
$\frac{\partial L}{\partial p_1} = -\log p_1 - 1 - 2\lambda_1 - \lambda_2 = 0$
$\frac{\partial L}{\partial p_2} = -\log p_2 - 1 - 4\lambda_1 - \lambda_2 = 0$
$\frac{\partial L}{\partial p_3} = -\log p_3 - 1 - 6\lambda_1 - \lambda_2 = 0$
$\frac{\partial L}{\partial p_4} = -\log p_4 - 1 - 8\lambda_1 - \lambda_2 = 0$
Now I was asked to formulate it in dual function. If I have understood it correctly, for dual I have to substitute all $p_i$ using $\lambda$. Thus:
$p_1 = e^{- 1 - 2\lambda_1 - \lambda_2}$
$ p_2 =e^{- 1 - 4\lambda_1 - \lambda_2} $
$ p_3 =e^{- 1 - 6\lambda_1 - \lambda_2} $
$ p_4 =e^{- 1 - 8\lambda_1 - \lambda_2} $
If I have done everything correctly until now, I should build the dual function:
$g(\lambda) = (e^{- 1 - 2\lambda_1 - \lambda_2})\log (e^{- 1 - 2\lambda_1 - \lambda_2}) - (e^{- 1 - 4\lambda_1 - \lambda_2}) \log (e^{- 1 - 4\lambda_1 - \lambda_2}) - (e^{- 1 - 6\lambda_1 - \lambda_2}) \log(e^{- 1 - 6\lambda_1 - \lambda_2}) - (e^{- 1 - 8\lambda_1 - \lambda_2}) \log (e^{- 1 - 8\lambda_1 - \lambda_2}) -\lambda_1( 2(e^{- 1 - 2\lambda_1 - \lambda_2}) + 4(e^{- 1 - 4\lambda_1 - \lambda_2}) + 6(e^{- 1 - 6\lambda_1 - \lambda_2}) + 8(e^{- 1 - 8\lambda_1 - \lambda_2}) - 4) - \lambda_2((e^{- 1 - 2\lambda_1 - \lambda_2}) + (e^{- 1 - 4\lambda_1 - \lambda_2}) + (e^{- 1 - 6\lambda_1 - \lambda_2}) + (e^{- 1 - 8\lambda_1 - \lambda_2}) - 1)$
Now I have to solve the dual analytically.
My question is, if I have done this correctly until now. If yes, how can I solve such a complicated function? Or is there a way to simplify it?
UPDATE
Use $v=e^{-\lambda_2 - 1}$ and $w=e^{-2\lambda_1}$
$p_1 = vw$
$p_2 = vw^2$
$p_3 = vw^3$
$p_4 = vw^4$
Now:
$ 2vw + 4vw^2 + 6vw^3 + 8vw^4 = 4$
$ v(2w + 4w^2 + 6w^3 + 8w^4) = 4$
$ v = \frac{2}{(w + 2w^2 + 3w^3 + 4w^4)}$
and for $g_2$:
$ vw + vw^2 + vw^3 + vw^4 = 1$
substituting with $ v = \frac{2}{(w + 2w^2 + 3w^3 + 4w^4)}$:
$\frac{2(w + w^2 + w^3 + w^4)}{(w + 2w^2 + 3w^3 + 4w^4)} = 1$
$2w + 2w^2 + 2w^3 + 2w^4 = w + 2w^2 + 3w^3 + 4w^4$
$w + w^3 - 2w^4 = 0$
Now resubstituting:
$e^{-2\lambda_1}+e^{-6\lambda_1}-2e^{-8\lambda_1}=0$