Conditional expectation of gaussian variable

109 Views Asked by At

Suppose $$X = QZ + m + V$$ where $Z \sim \mathcal{N}(0, I_K)$, $Q \in \mathbb{R}^{M\times K}$, $m \in \mathbb{R}^M$ and $V \sim \mathcal{N}(0, \sigma^2 I_M)$. $Z$ and $V$ are independent.

I'm having trouble with one of the steps in a proof for the following expression for the expected value of $Z$ given $X$. $$E[Z\mid X=x] = (Q^TQ + \sigma^2 I_K)^{-1}Q^T(x-m).$$

The step I don't understand is: $$E[Z\mid X=x] = E[Z(X-m)^T]E[(X-m)(X-m)^T]^{-1}(x-m).$$

Is there a Bayes rule for expected values I can use? Can someone explain this step to me?

Edit: This is in the context of probabilistic principle component analysis. The data points $x$ are modeled as if they originated from a linearly transformed lower dimensional random variable ($Z$) plus high dimensional gaussian noise ($V$). So $M \geq K$ and $Q$ is assumed to be full rank.

1

There are 1 best solutions below

4
On BEST ANSWER

Suppose $$ \left[ \begin{array}{c} T \\ U \end{array} \right] \sim \mathcal N\left( \left[ \begin{array}{c} 0 \\ 0 \end{array} \right], \left[ \begin{array}{cc} A & B \\ B^T & C \end{array} \right] \right) \tag 1 $$ where $A\in \mathbb R^{p\times p},$ $B\in\mathbb R^{p\times q},$ $C\in\mathbb R^{q\times q}$ and the big matrix in $(1)$ is positive-definite. Then \begin{align} T\mid U \sim\mathcal N\left( BC^{-1} U , A - BC^{-1}B^T \right). \end{align}

Apply this in the case where $T=Z,$ $U=X-m,$ $A=I_K,$

\begin{align} \require{cancel} B = {} & \operatorname{cov}(Z, QZ+V) \\[10pt] = {} & \operatorname{cov}(Z, QZ) + {} \cancelto0{\operatorname{cov}(Z,V)\,\,\,} \\[10pt] = {} & \operatorname{cov}(Z,Z)Q^T \\ & \text{Here the $Q$ gets transposed and gets} \\ & \text{pulled out on the right, not the left.} \\[10pt] = {} & Q^T, \\[15pt] \text{and } C = {} & QQ^T + \sigma^2 I_M. \end{align}