I was wondering how to show entropy of the zeta distribution.
It is: $$ H_\mathrm{zeta}(X) = \sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \log(k^s \zeta(s))$$
The entropy of the zipf distribution is:
$$ H_\mathrm{zipf}(X) = \frac{s}{H_{N,s}} \sum_{k=1}^N \frac{\ln(k)}{k^s} + \ln(H_{N,s})$$
The zipf distribution with parameter $N = \infty$ is the zeta distribution.
Then, taking the limit $H_\mathrm{zipf}(X)$ as $N$ goes to $\infty$,
$$ \lim_{N \to \infty} \frac{s}{H_{N,s}} \sum_{k=1}^N \frac{\ln(k)}{k^s} + \ln(H_{N,s}) $$
$$ = \frac{s}{\zeta(s)} \sum_{k=1}^\infty (1/k^s) \ln(k) + \ln(\zeta(s))$$
$$ = \sum_{k=1}^\infty (1/k^s) \ln(k^s) (1/\zeta(s)) + \ln(\zeta(s))$$
$$ = \sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s) + \ln(\zeta(s))\cdots\text{(1)}$$
Note that I have used the identity: $\lim_{N \to \infty} H_{N,s} = \zeta(s)$
In the last equation, equation (1), I was unable to combine the natural log terms because it seems $\ln(\zeta(s))$ is outside of the summation. How do I proceed?
Thanks.
Since $\sum\limits_{k=1}^\infty \frac{1/k^s}{\zeta(s)} =1$, (1) is $$\left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s)\right) + \ln(\zeta(s))=\left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s)\right) + \left(\sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)}\right) \ln(\zeta(s)), $$ that is, as desired, $$ \sum_{k=1}^\infty \frac{1/k^s}{\zeta(s)} \ln(k^s\zeta(s)). $$