How to find the subspace on which a multivariate normal distribution is concentrated?

311 Views Asked by At

Let $(X_1,X_2,X_3)^T$ be a multivariate normal distribution which is singular (that is, its variance matrix has determinant $0$).

How can we find the subspace $U \subset \mathbb{R}^3$ with dimension strictly less than $3$ such that $P( (X_1, X_2,X_3)^T \in U) = 1?$.

1

There are 1 best solutions below

1
On

Short answer: Take $3$ independent samples from this distribution $a_1,a_2,a_3\in R^3$. With probability $1$, $U=\text{span}(a_1,a_2,a_3)$.

Longer answer to determine $U$: Stack $a_1,a_2,a_3$ next to each other to obtain matrix \begin{equation} A=[a_1~a_2~a_3] \end{equation} Since distribution is singular, $A$ will be rank definition with rank $\text{dim}(U)=r<3$ with probability $1$. Take the (skinny) singular value decomposition of $A=V_1 \Sigma V_2^*$ where diagonal singular value matrix is $\Sigma\in R^{r\times r}$. Then, subspace $U$ obeys \begin{equation} U=\text{range}(V_1) \end{equation} where $\text{range}(V_1)$ returns span of columns of $V_1$. There are exactly $r$ columns of $V_1$ which are orthonormal. This nicely characterizes $U$.