I am currently working on an information theory problem where I have to bound the divergence between two distributions.
The divergence can be simplified to:
$$\sum_{k=0}^N \ {N\choose k} x^k(1-x)^{N-k} \Big[ \log( x^k(1-x)^{N-k})+\log(N+1)+\log{N\choose k} \Big]$$
where $x \in [0,1]$
And I have to show that this expression is less than $\log(N+1).$
Does anyone know how to proceed?
I am assuming $n = N$. Essentially, you need to show that $$S = \sum_{k=0}^{N} \dbinom{N}k x^k (1-x)^{N-k} \log\left(\dbinom{N}k x^k (1-x)^{N-k}\right) \leq 0$$ Now for $x \in (0,1)$, we always have $$\dbinom{N}k x^k (1-x)^{N-k} < 1$$ since the sum over $k$ of these add upto one and each term is positive. Hence, we have $$\log\left(\dbinom{N}k x^k (1-x)^{N-k}\right) < 0$$ Hence, each term in the summation of $S$ is negative and hence $S$ is negative.