Joint entropy maximization with a constraint

145 Views Asked by At

So I have 2 random variables $X$ and $Y$, where $X$ can take on values $\{0,1,2,3\}$ and Y can take on values $\{0,1,2,3,4\}$. I need to maximize $H(X,Y)$ subject to the constraint that $P(X\neq Y)=0.5$. This also gives $P(X=Y)=0.5$.

Now, we can't assume that $X$ and $Y$ are uniform (the maximal entropy distribution without constraints) because of the given constraint. However, we can add another random variable $U$ such that it is $0$ when $X \neq Y$ and $1$ when $X=Y$.

My professor says that I can then optimize for the 2 cases separately by taking advantage of the fact that $H(X,Y)=H(X,Y,U)$. However, I am not really sure how to go about finding $H(X,Y,U)$ without knowing the marginal PMFs of $X$ and $Y$. What would be a step in the right direction to solving this? Note that $$H(U)=h_b(0.5)=1$$