I'm writing a paper on Information Theory and I can't get my head around this task:
I'd like to prove that the maximum value of the Shannon entropy function $H$ is reached when each event in the set of probabilities $Ps=\{P(x_1), P(x_2), ..., P(x_n)\}$ has the same value.
The $H$ function is defined like this:
$$ H(S)=\sum\limits_{i=1}^{card(S)}P(x_i)*(-\log_2(P(x_i))) $$
I could only prove this with $card(S)<=2$ but could not find any technique to do it for $card(S) = N$.
I think that a possible solution would be solving it with a proof by induction using $card(S)$ (the length of $S$) as our parameter.
There are many ways to do this, but perhaps one of the easiest is via Jensen's inequality. Let $n$ indicate the cardinality of $S$. Then, write $$ \begin{align} H(S) & = \sum_x p(x) (-\log p(x)) \\ & = \sum_x p(x) \log \frac{1}{p(x)}\\ & \le \log \sum_x p(x) \frac{1}{p(x)}\\ & =\log n \end{align} $$ The inequality uses Jensen's inequality and that $\log$ is a concave function. It is easy to check that the uniform distribution, $p(x)=1/n$ for all $x$, achieves this upper bound.