channel capacity of a channel with a state

180 Views Asked by At

I am trying to understand the solution of question 2, part 1 in this question sets. The output is $Y=X \circ Z$, where $Z$ is Bernoulli noise, $\circ$ is sum mod 2, $X$ is a Bernoulli input. $Z$ probabilities change based on the channel state, which is represented by the random variable $S$. It asks for the capacity of the channel when both the encoder and the decoder have access to the state sequence. I don't understand how the maximum of the mutual information is achieved when the input, $X$~$Bernoulli(.5)$ as shown in the screen shot?

enter image description here

1

There are 1 best solutions below

2
On

There is an unimportant typo in the first line, instead of $Y=X+S+Z$ it should be $Y=X+Z$

Now consider $$ C= \max_{P(X|S)}I(X;Y)= \max_{P(X|S)} H(X+Z|S) - H(Z|S)$$

The second term is not influenced by the distribution of $X|S$, hence it's a constant.

Then we want to maximize the first term, which is bounded by $H(Y|S)\le H(Y) \le 1$ bit (because it's a binary variable). It remains to check that this bound is a maximum, i.e. it's attainable by some input distribution. And it indeed is, because if $P(X|S)$ is uniform (regardless of $S$), then $P(Y|S)$ is also uniform.

Then we've found the capacity. What remains is simply to express $H(Z|S)$ in terms of the given probabilities, and that's done in the last two lines.