How do we solve for the $n\times n$ variance - covariance matrix?

88 Views Asked by At

Suppose that $x_1, x_2, \dots, x_n$ follow a $n-$ dimensional multivariate normal with mean and variance - covariance matrix as below

$$(\mu_n, \Sigma_n) =\begin{bmatrix} \begin{bmatrix}\mu_1\\ \mu_2 \\ \vdots\\ \mu_n\end{bmatrix} , \begin{bmatrix}\sigma_1^2 & \sigma_{1,2} & \sigma_{1,3} & \cdots & \sigma_{1,n} \\ \sigma_{2,1} & \sigma_2^2 & \sigma_{2,3} & \cdots & \sigma_{2,n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \sigma_{n,1} & \sigma_{n,2} & \sigma_{n,3} & \cdots & \sigma_n^2\end{bmatrix} \end{bmatrix}$$

In case where $n=2$ in order to calculate the dererminant of $\Sigma$ and find the inverse variance - covariance matrix we have the following formulas

$$|\Sigma_2| = \sigma_1^1\sigma_2^2(1-\rho^2)$$

and hence

$$\Sigma_2^{-1}=\frac{1}{\sigma_1^1\sigma_2^2(1-\rho^2)}\begin{bmatrix}\sigma_1^2 & -\rho\sigma_1\sigma_2 \\ -\rho\sigma_1\sigma_2 & \sigma_2^2\end{bmatrix}$$

Assuming that the $n\times n$ variance co-variance matrix is positive semi-definite and the inverse matrix exists how do we solve for the determinant of $\Sigma_n$ and the inverse $n\times n$ variance - covariance matrix $\Sigma_n^{-1}$?

Let me make a try. Assuming that the inverse of the covariance matrix exists, then

$$|\Sigma| = \begin{vmatrix}\sigma_1^2 & \rho_{1,2}\sigma_1\sigma_2 & \rho_{1,3}\sigma_1\sigma_3 & \cdots & \rho_{1,n}\sigma_1\sigma_n \\ \rho_{2,1}\sigma_2\sigma_1 & \sigma_2^2 & \rho_{2,3}\sigma_2\sigma_3 & \cdots & \rho_{2,n}\sigma_2\sigma_n \\ \vdots & \vdots & \vdots & \ddots & \vdots\\ \rho_{n,1}\sigma_n \sigma_1 & \rho_{n,2}\sigma_n\sigma_2 & \rho_{n,3}\sigma_n\sigma_3 & \cdots & \sigma_n^2 \end{vmatrix} = Diag(\Sigma)^{-1/2} A Diag(\Sigma)^{-1/2}=\\=\sigma_1^2\sigma_2^2\cdots\sigma_n^2|A|$$

where

$$Diag(\Sigma)^{-1/2} = \begin{bmatrix}\sigma_1 & 0 & 0 & \dots & 0 \\ 0 & \sigma_2 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0& 0 & \cdots & \sigma_n \end{bmatrix}$$

$$A = \begin{bmatrix}1 & \rho_{1,2} & \rho_{1,3}& \cdots & \rho_{1,n} \\ \rho_{2,1} & 1 & \rho_{2,3} & \cdots & \rho_{2,n} \\ \vdots & \vdots & \vdots & \ddots & \vdots\\ \rho_{n,1} & \rho_{n,2} & \rho_{n,3} & \cdots & 1 \end{bmatrix}$$

Could anyone help from this step and beyond to find the co-factor matrix? All I need is a recursive formula

1

There are 1 best solutions below

0
On

Foreward: I say covariance matrix because its shorter and is synonymous to covariance-variance to the best of my knowledge.

The inverse of a covariance matrix (if it exists) is called a precision matrix. For a univariate distribution our explicit definition is: $$p = \frac{1}{\sigma^2}$$ However, this does not translate to multivariate distributions. Let $\Sigma$ denote our covariance matrix for $x_1, x_2, \ldots, x_n$, and $P$ denote our precision matrix. We know that $P = \Sigma^{-1}$, so we can roughly define:

$$P = \frac{1}{\det(\Sigma)} \text{Adj}(\Sigma)$$ Where $\text{Adj}(\Sigma)$ is the transpose of the cofactor matrix of $\Sigma$. Also, the covariance matrix $\Sigma$ is defined as:

$$\Sigma = \begin{pmatrix} \text{Cov}(x_1, x_1) & \text{Cov}(x_1, x_2) & \ldots & \text{Cov}(x_1,x_n) \\ \text{Cov}(x_2,x_1) & \text{Cov}(x_2,x_2) & \ldots &\text{Cov}(x_2,x_n) \\ \vdots & \vdots & \ddots & \vdots \\ \text{Cov}(x_n,x_1) & \text{Cov}(x_n, x_2)& \ldots & \text{Cov}(x_n, x_n) \end{pmatrix} $$ Not what you have written down. They are very, very different, so maybe you mean something else but you should change this word.

Outside of this, there are not many things that we can say about it without much more complex calculations, since each element of a precision matrix is not particularly saying anything. Wikipedia has a page on partial correlations here, which mention precision matrices and the properties of the elements but it won't do you much good. The only other formula I could find is here, with the density of a normal distribution. Generally, the explicit formula for an inverse is the easiest to calculate, and we can do it in time complexity $\mathcal{O}(n^3)$ for a $n$ by $n$ matrix (I believe).

In sum, there is no known formula for the inverse of a covariance matrix, and I don't believe it's an open problem, which implies someone has proven it to be impossible to find.