Rate distortion function with infinite distortion

785 Views Asked by At

I am working through the problems in Elements of Information Theory by Cover and Thomas and have come across the following problem I couldn't answer.

The problem is to find the rate distortion function $R(D) = \min_{p(\hat x | x)} I(X;\hat X)$ for $X$ ~ Bernoulli$(\frac{1}{2})$ and distortion $d(x, \hat x)$ defined as

\begin{cases} 0, & \text{if $x=\hat x$} \\ 1, & \text{if $x=1$ and $\hat x=0$} \\ \infty, & \text{if $x=0$ and $\hat x=1$} \end{cases}

The solution I have defines $p(x,\hat x)$ as:

$$ \begin{bmatrix} \frac{1}{2} & 0 \\ D & \frac{1}{2}-D \\ \end{bmatrix} $$

I cannot figure out how this distribution was derived. I understand that $d(0,1)=\infty$ implies that $p(0,1)=0$, assuming that $D$ is finite. But I cannot derive the other three entries... Can someone offer a hint?

1

There are 1 best solutions below

0
On

Apologies in advance for not being able to format this properly.

The definition of D is the expectation value, E, of the distortion measure. E[d(x, x^)] = p(1|0) * 1 = p(1|0)= D. Now, because X is Bernoulli with probability 1/2, we have that p(0) = 1/2 = p(0|0) + p(0|1) = p(0|0). Finally p(1) = 1/2 = p(1|0) + p(1|1) = D + p(1|1), so p(1|1) = 1/2 - D.

This should take care of all the entries.