Maximum entropy of the sum of two vectors

195 Views Asked by At

Consider two identically distributed independent random vectors $X$ and $Y$ of dimension $n$ (assume $n$ is large). Both $X$ and $Y$ have only non-negative integer entries less than or equal to $n$ and $H(X)= H(Y) = n$.

What is the maximum possible value of $H(X + Y)$?

In particular, how close can it get to $2n$?

My working so far

If $X$ and $Y$ are random $0/1$ vectors then $H(X+Y) = 3n/2$ I believe. This is the highest value for $H(X+Y)$ I have managed to find so far.

1

There are 1 best solutions below

3
On

There is a well known bound on $H(X+Y)$ i.e., $\max{\{H(X),H(Y)\}}\leq H(X+Y) \leq H(X)+H(Y)$. Hence the maximum possible value of $H(X+Y)$ is $H(X)+H(Y)$. Equality holds iff $X=Y$. Hence $H(X+Y)\leq 2n$, and it can come close to $2n$ by picking same value for $X$ and $Y$ always (to be more precise $X=Y$ almost surely).

For proof of above bound, one can refer to http://www2.isye.gatech.edu/~yxie77/ece587/SumRV.pdf