Challenges in Implementing the Blahut–Arimoto Algorithm for channel capacity computation

43 Views Asked by At

I've read the Blahut–Arimoto Algorithm from the original paper by Blahut and Raymond Yeung's book, along with other sources. However, I'm having trouble understanding the computation of channel capacity in the algorithm, particularly in Fig.1 of Blahut's paper (corresponding to Theorem 1). Specifically, when $P_{j|k}^* = 0$, how can we calculate $P_j^* = \frac{exp(\sum_k Q_{k|j}log(P_{j|k}^*))}{\sum_j exp(\sum_k Q_{k|j}log(P_{j|k}^*))}$?

The first expression for $P_{j|k}^*$ assumes it is non-zero, derived with the assumption and a well-known IT inequality. The second is derived using KKT conditions. In some implementations (e.g., here), they assume $0 \cdot \log(0) = 0$, but I believe this is incorrect.

Consider the testing scenario with $Q=P\{Y|X\}=\begin{bmatrix} 1 & 0\\ 0.5 & 0.5\\ 0&1 \end{bmatrix}$. The capacity of this channel should be 1.5, with the achieved distribution being [1/3 1/3 1/3]. However, the algorithm does not yield these values.

Any insights or corrections would be appreciated.