Capacity of Z Channel

2.9k Views Asked by At

Calculating the capacity of the Z channel (binary asymmetric channel) here, the entropy $ H(Y)$ isn't supposed to be $ H(Y)=H(a+(1-a)p,(1-a)(1-p))$ ? What's the reason for having $H(Y)=H((1-a)(1-p))$ ?

1

There are 1 best solutions below

6
On BEST ANSWER

There is a common abuse of notation here: $H(\cdot)$ can mean two very different things.

$H(X)$ (the argument is a random variable) means the entropy of a random variable $X$ (with some given density).

$H(a)$, when the argument is a scalar (more specifically, a real number in $[0,1]$), means the "binary entropy", that is, the entropy of a Bernoulli variable with paramenter $a$ ($P[X=1]=a)$. Hence $H(a) = -a \log(a) -(1-a) \log(1-a)$

In the deduction , $Y$ is a Bernoulli variable, hence $H(Y)=H(p)$ where the two sides of the equation use (perhaps confusingly) the two different notations, the right side is the binary entropy, and $p=P(Y=1)$

Update: Using (for avoiding confusion) the notation $H(X)$ for the entropy of a random variable and $h_b( p)$ the binary entropy function. In the Z channel case we have a random variable $Y$ (output) which is a Bernoulli variable . Hence $H(Y) = h_b(t)$ where $t=P(Y=1)$ (I use the variable name $t$ to avoid -another- confusion with $p=$ probability of channel crossover).

Now, $$t=P(Y=1)= P(X=1 \cap {\rm no crossover})=(1-\alpha)(1-p)$$

Hence $$H(Y)= h_b(t)=h_b((1-\alpha)(1-p))$$