Minimizing \begin{align*} F(p)&= \sum \limits_{n=1}^\infty p_n \log_2 \left(n^2 p_n \right) + \log_2 \left( \frac{\pi^2}{6} \right) \end{align*} subject to \begin{align*}\sum \limits_{n=1}^\infty p_n = 1\end{align*}
I tried $$\mathcal L \left(p_1, p_2, ..., \lambda \right) = \sum \limits_{n=1}^\infty p_n \log_2 \left(n^2 p_n \right) + \log_2 \left( \frac{\pi^2}{6} \right) - \lambda \left( \sum \limits_{n=1}^\infty p_n - 1 \right) $$
$$\frac{\partial \mathcal L}{\partial p_i} = \frac{1}{\ln(2)} + \log_2(i^2 p_i) - \lambda = 0$$ $$\frac{\partial \mathcal L}{\partial \lambda} = 1 - \sum_{n=1}^\infty p_n = 0$$
$$ \frac{\partial \mathcal L}{\partial p_i} = 0 \text{ gives } p_i = \frac{2^\lambda}{e i^2}$$
Imposing the constraint $$\sum_{n=1}^\infty p_n = 1$$
gives $$p_i = \frac{6}{\pi^2 i^2} \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, (1)$$
This looks promising but I haven't used Lagrange multipliers method before. As I understand, $(1)$ gives me the critical point of $F(p)$. How do I know this is the minimum value?
Assuming your calculations are correct. The solution obtained is indeed a minimum. Since the objective function is convex, and the equality constraint is linear. Lagrange multiplier method is guaranteed to give minima.