how to find the probability density function that maximizes the entropy under the constraint $E(X)=\sum_{n=0}^{\infty}{n\cdot p(n)}=A$?

Question

I'm approaching this with Lagrange multipliers, I have defined

$$L(p(0),...,p(n),\lambda)=H(X)-\lambda\cdot(E(X)-A)=-\sum_{n=0}^\infty{p(n)\cdot{log(p(n))}}-\lambda(\sum_{n=0}^{\infty}{n\cdot p(n)}-A)$$

Did i define the Lagrange function correctly? if yes I'm having trouble calculating the derivatives and finding $\lambda$.

Answer 1

You actually have

$$L(p_0,...,p_n, \cdots ; \lambda,\beta)=H(X)-\lambda\, (E(X)-A)-\beta \, ( \sum p_n -1)$$

Then notice that your "variable" is actually the sequence (or, if you prefer the infinite-dimensional vector) ${\bf p}=[p_0, p_1 , \cdots, p_n \cdots ]$. And both $E[X] $ and $H(X)$ are functions (functionals) of that variable.

Then you have to set all the derivatives (or, if you prefer, the infinite-dimensional gradient) to zero:

$$ \frac{ \partial L(p_0,...,p_n, \cdots ; \lambda,\beta)}{\partial p_i}=0 \quad \forall i=0, 1 \cdots$$

Can you go on from here?

Edit: You get, for the critical point, the equation

$$-(log(p_n)+1)-\lambda \cdot n -\beta=0$$

which can be written as $$ p_n = b \, e^{-an}$$ where $a,b$ are (positive) numbers that can be obtained from the two restrictions. This is a geometric distribution.