Strict concavity of conditional mutual information

123 Views Asked by At

Let $I(X;Y\vert Z)$ be the conditional mutual information between random variables $X$, $Y$, and $Z$. We know that $I(X;Y\vert Z)$ is the mutual information between $X\vert Z$ and $Y\vert Z$, and also, based on Chapter 3, Lemma 3.3 in "Physical-Layer Security: From Information Theory to Security Engineering" by Bloch et al., we know that $I(X;Y\vert Z)$ is a "concave" functional of $p_X(x)$.

My question is, provided that we have the degardedness condition, i.e., $p(x,y,z) = p(z\vert y) p(y\vert x) p_X(x)$, do we have the "strict concavity" of $I(X;Y\vert Z)$ in $p_X(x)$?

1

There are 1 best solutions below

2
On

Let $p$ be a length-$n$ probability vector, $H(p)$ be the entropy of $p$, and $W$ be a stochastic matrix. In general, one does not get strict concavity of the mutual information $$ I(p,W) = H(pW) - \sum_{i=1}^n p_i H(e_i W), $$ where $e_i$ is the $i$-th standard basis vector.

Since $H(\cdot)$ is strictly convex and the second term is linear in $p$, the mutual information $I(p,W)$ will be strictly convex if $W$ has rank $n$.

On the other hand, $I(p,W)$ will not be strictly convex if the null space of $W^T$ contains a vector $q$ that sums to 0. This is because $I(p+sq,W)$ will be independent of $s$ when $p$ is uniform and $s$ is sufficiently small.

This settles the degraded case because $I(X;Y|Z) = I(X;Y)-I(X;Z)$ for the Markov chain $X-Y-Z$ and subtracting $I(X;Z)$ cannot make it concave. Also, choosing $Z$ to be constant settles the general case for $I(X;Y|Z)$.