How do I extract information about the standard deviation from the following variance matrix

84 Views Asked by At

Let $X$ = $\begin{pmatrix}X_{1} \\X_{2} \end{pmatrix}$ with

$$ X \sim N\left(\begin{pmatrix}5 \\ 2 \end{pmatrix}, \begin{pmatrix} 36&-30\\ -30&26 \end{pmatrix} \right)$$

I have been asked to decompose $Var[X]$ into a product of a lower triangular matrix and it's transpose: $$\mathrm{Var}[X] = LL^{T}$$

for which I obtained:

$$\mathrm{Var}[X] = \begin{pmatrix} -6&0\\ 5&1 \end{pmatrix}\begin{pmatrix} -6&5\\ 0&1 \end{pmatrix}$$

I am now being asked to write down the formula below and replace the *'s with appropriate numbers:

$$\begin{pmatrix}X_{1} \\X_{2} \end{pmatrix} = \begin{pmatrix}* \\* \end{pmatrix} + \begin{pmatrix} *&*\\ *&* \end{pmatrix}\begin{pmatrix}Z_{1} \\Z_{2} \end{pmatrix}$$
where $Z_{1}$ and $Z_{2}$ are two independent $N(0,1)$ random variables.

I understand where the formula is derived from as it is simply a vector and matrix representation of the Z-score standardisation formula given by:

$$Z_{1} = \frac{X_{1}-\mu_{1}}{\sigma_{1}}$$

and by rearranging, we get:

$$X_{1} = \mu_{1} + Z_{1}\sigma_{1}$$ and $$X_{2} = \mu_{2} + Z_{2}\sigma_{2}$$

Then, by using vector and matrix notation, we can re-write both random variables as:

$$\begin{pmatrix}X_{1} \\X_{2} \end{pmatrix} = \begin{pmatrix}5 \\2 \end{pmatrix} + \begin{pmatrix} \sigma_{1,1}&\sigma_{1,2}\\ \sigma_{2,1}&\sigma_{2,2} \end{pmatrix}\begin{pmatrix}Z_{1} \\Z_{2} \end{pmatrix}$$

I am struggling to find the values of $\sigma_{1,1}$, $\sigma_{1,2}$, $\sigma_{2,1}$, $\sigma_{2,2}$.

Are they the just the values of the conditional variance?

1

There are 1 best solutions below

0
On BEST ANSWER

The matrix you are looking for is $L$!

This is one of the problems where you are led to make a guess and check it. Let us guess that $X = \mu + LZ$ where $\mu = [\mu_1, \mu_2]^T$ and check that both the mean vector and the covariance matrix agree with the values stated at the top of your question. This is sufficient because a Gaussian distribution is completely determined by its mean and covariance matrix.

Note that the distribution of the $Z$ vector is

$$ \begin{pmatrix} Z_1 \\ Z_2 \end{pmatrix} \sim \mathcal{N}\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\right) $$

i.e. $\mathbb{E}(Z) = 0$ and $\mathrm{Var}(Z) = I$.

Start with the mean

$$ \mathbb{E}[X] = \mathbb{E}[\mu + LZ] = \mathbb{E}[\mu] + \mathbb{E}[LZ] = \mu + L \, \mathbb{E}[Z] = \mu + L \cdot 0 = \mu $$

which agrees with what you have already found.

Next, the covariance matrix

$$ \begin{align} \mathrm{Var}[X] &= \mathrm{Cov}(X, X) \\ &= \mathrm{Cov}(\mu + LZ, \mu + LZ) \\ &= \mathrm{Cov}(\mu + LZ, \mu) + \mathrm{Cov}(\mu + LZ, LZ) \\ &= \mathrm{Cov}(\mu, \mu) + \mathrm{Cov}(LZ, \mu) + \mathrm{Cov}(\mu, LZ) + \mathrm{Cov}(LZ, LZ) \\ &= 0 + 0 + 0 + \mathbb{E}[(LZ - 0) (LZ - 0)^T] \\ &= \mathbb{E}[LZZ^TL^T] \\ &= L \, \mathbb{E}[ZZ^T] \, L^T \\ &= L \, \mathrm{Var}[Z] \, L^T \\ &= L L^T. \end{align} $$

Thus, our guess that $X = \mu + LZ$ was correct and therefore

$$ \begin{pmatrix}X_{1} \\X_{2} \end{pmatrix} = \begin{pmatrix}5 \\2 \end{pmatrix} + \begin{pmatrix} -6&0\\ 5&1 \end{pmatrix}\begin{pmatrix}Z_{1} \\Z_{2} \end{pmatrix}. $$


Remark 1: Your attempted solution is fairly close (in particular, it found the mean correctly), but it misses an important fact: that the two components of $X$ may depend on each other. The issue is that in your equations $X_1$ depends on $Z_1$ but not on $Z_2$ and similarly for $X_2$. It is easy to avoid such mistakes by writing equations in terms of vectors and matrices rather than components.


Remark 2: Knowledge provided by this exercise is of great practical value. It suggests a very simple algorithm to generate samples from multivariate Gaussian distribution with arbitrary mean and covariance.