How to compute the conditional expectation of a jointly multivariate normal distribution?

108 Views Asked by At

In the paper Functional Linear Discriminant Analysis, on page 7, they use the EM algorithm. The model involves random variables $\gamma\sim\mathcal{N}_m(0, \Gamma)$ and $\epsilon\sim\mathcal{N}_n(0, \sigma²I)$ and is of the form $$\mathbf{Y}=S(\beta + \gamma) + \epsilon$$, where $S$ is a design matrix, $\beta$ is the parameter and $\mathbf{Y}$ is the regression target. They then try to compute the MLE of $\Gamma, \sigma²$ and $\beta$ via the EM algorithm, treating $\gamma$ as a latent variable. Clearly, the joint log likelihood is $$l(\sigma², \Gamma, \beta)\propto-\frac{||\mathbf{Y} - S(\beta+\gamma)||²}{\sigma²}-\gamma^T\Gamma^{-1}\gamma-\log(|\Gamma|)-n\log(\sigma²)$$ However, they then compute the expectation of $\gamma$ for the E-step as $$\mathbb{E}[\gamma|\mathbf{Y}, \beta, \Gamma, \sigma²]=(\sigma²\Gamma^{-1} + S^TS)^{-1}S^T(\mathbf{Y}-S\beta)$$

I cannot move ahead after computing $S\gamma = \mathbf{Y}-S\beta-\epsilon$. I know that this means $S\mathbb{E}[\gamma|\mathbf{Y},\beta,\Gamma,\sigma²]=\mathbf{Y}-S\beta$, but how do I transfer $S$ to the RHS?

1

There are 1 best solutions below

0
On BEST ANSWER

Nvm I solved it by writing the joint density $f_{Y,\gamma}(\gamma)$ and representing the terms involving $\gamma$ as another multivariate normal distribution. Then I computed $f_Y(Y)$ and showed that the terms in the exponential of $f_{\gamma|Y}(\gamma)=\frac{f_{Y,\gamma}(\gamma)}{f_Y(Y)}$ involving $\gamma$ form another multivariate normal with mean $(\sigma²\Gamma^{-1}+S^TS)S^T(Y-S\beta)$, which solves the problem.