Sequential Information Projection (Repeated information property)

58 Views Asked by At

The lecture note Here at last paragraph of page 136 explained the concept of 'repeated information projection' without much proof. I am wondering how can I prove the following property. Any help would be appreciated

Repeated information projection

If $Q_1$ is the information projection of $P$ onto $\Lambda_1$ (that is,

$$ D\left(Q_1 \parallel P \right) = \inf_{Q \in \Lambda_1}D\left(Q \parallel P\right). $$

Also, the $Q_2$ is the information projection of $Q_1$ onto $\Lambda_1 \cap \Lambda_2$ (that is, $$ D\left(Q_2 \parallel Q_1 \right) = \inf_{R \in \Lambda_1 \cap \Lambda_2}D\left(R \parallel Q_1\right). $$

Then $D\left(Q_2 \parallel P \right) = \inf_{S \in \Lambda_1 \cap \Lambda_2} D\left(S \parallel P \right)$, too.