Expansion of the Multivariate Gaussian Distribution formula

1.6k Views Asked by At

I'm reading through these notes that claim that if we expand the log of the multivariate gaussian distribution we get the following:

$$log(N(x_n|\mu_k, \Sigma_k)) = -\frac{1}{2} (x_n - \mu_k)^T \Sigma_k^{-1}(x_n - \mu_k) -\frac{1}{2} log|\Sigma_k| -\frac{D}{2} log(2\pi)$$

This part makes sense, however upon expansion it says we have this:

$$log(N(x_n|\mu_k, \Sigma_k)) = \mu_k^{T} \Sigma_k^{-1}x_n - \frac{1}{2} \mu_k^T \Sigma_k^{-1}\mu_k$$

I'm not sure how expanding the top function results in the bottom one, I'm fairly certain $-\frac{D}{2} log(2\pi)$ was removed because eventually we want to maximize over k but i'm not sure about the rest of it...

any help would be appreciated.

Edit: The formula for a multivariate gaussian is:

$$N(x_n|\mu_k, \Sigma_k) = \frac{1}{(2\pi)^{\frac{D}{2}} |\Sigma|^{\frac{1}{2}}} e^{- \frac{1}{2} (x_n - \mu_k)^T \Sigma^{-1}(x_n - \mu_k)}$$

1

There are 1 best solutions below

3
On

Ignoring all the subscripts, and writing $A:=\Sigma^{-1}$, expand as follows: $$(x-\mu)^TA(x-\mu)=x^TAx-x^TA\mu-\mu^TAx+\mu^TA\mu$$ All the terms on the RHS are scalars since they have dimension $(1\times D)\times (D\times D)\times (D\times 1)$. Take the transpose of the second term and we see it's equal to the third term (since the covariance matrix $A$ is symmetric). This now simplifies to your form if we're throwing away (i.e., treating as constant) any term that doesn't involve $\mu$. (This would be appropriate if you were trying to maximize the log likelihood over $\mu$.)