ln() and product confuse me

Question

I have this function:

$p(x_n;\mu) = \frac{1}{(2\pi)^{\frac{l}{2}}|\Sigma|^{\frac{1}{2}}}\exp\left(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right)$

And I have to compute this: $L(\mu) = \ln\left( \prod\limits_{n=1}^{N} p(x_n;\mu)\right)$ which should be equal to:

$-\frac{N}{2}\, \ln( \space (2\pi)^l|\Sigma|\space) \space- \space \frac{1}{2} \space \sum\limits_{n=1}^{N}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)$

Note: $\Sigma$ is a covariance matrix.

---

From my attempt I have:

$\frac{1}{(2\pi)^{\frac{l}{2}}|\Sigma|^{\frac{1}{2}}} = \ln1 - \ln((2\pi)^{l/2}|\Sigma|^{1/2}) = - \ln((2\pi)^{l/2}|\Sigma|^{1/2}) = -\frac{L}{2}\ln2\pi - \frac{1}{2}\ln|\Sigma|$

and:

$\ln(\exp(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu))) = -\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)$

but when I put everything together with the product, it doesn't work out, squares appear and it doesn't feel right...Can you help me please understand?

Answer 1

We are given that

$$p(x_n;\mu):= \frac{1}{(2\pi)^{\frac{l}{2}}|\Sigma|^{\frac{1}{2}}}\exp\left(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right)$$

Since $p$ is real valued we should be able to apply ordinary log rules within reason. These are:

1. $\ln(xy)=\ln(x)+\ln(y)$ which generalises to $\ln\left(\prod_i x_i\right)=\sum_i \ln(x_i)$ for finite sums and products.
2. $\ln(a^b)=b\ln(a)$
3. $\ln(\exp(y))=y$

Now we want to find:

$$L(\mu)=\ln\left(\prod_{n=1}^N p(x_n;\mu)\right)=\sum_{n=1}^N\ln(p(x_n;\mu))\quad \text{using point }1. $$

Now we find $\ln(p(x_n;\mu)):$ \begin{align} \ln(p(x_n;\mu))&= \ln\left((2\pi)^{-l/2}|\Sigma|^{-1/2}\exp\left(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right)\right)\\ &=\ln\left(\left[(2\pi)^l|\Sigma|\right]^{-1/2}\right)+\ln\left(\exp\left(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right)\right)\\ &=-\frac{1}{2}\ln\left((2\pi)^l|\Sigma|\right)+\left(-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right)\\ &=-\frac{1}{2}\ln\left((2\pi)^l|\Sigma|\right)-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu) \end{align} We used points $2$ and $3$ in going from line $2$ to $3$ above. Now to do the sum, note the first term doesn't depend on $n$

\begin{align} \sum_{n=1}^N\ln(p(x_n;\mu))&=\sum_{n=1}^N \left[-\frac{1}{2}\ln\left((2\pi)^l|\Sigma|\right)-\frac{1}{2}(x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\right]\\ &=-\frac{N}{2}\ln\left((2\pi)^l|\Sigma|\right)-\frac{1}{2}\sum_{n=1}^N (x_n-\mu)^T\Sigma^{-1}(x_n-\mu)\\ \end{align}

Which is what you want.