Since the variance matrix is the expected value of a dyadic tensor, why is it not singular? Which is the probabilistic property behind that?

355 Views Asked by At

I will try to explain better my annoying doubt.

The variance matrix (or covariance matrix, according to an alternative notation) $\Sigma_v \in \mathbb{R}^{n\times n}$ of the vector random variable $v\in \mathbb{R}^n$ is defined as: $$\mathrm{V}[v]=\mathrm{E}[(v-\mathrm{E}[v])(v-\mathrm{E}[v])^T]$$ which is the expectation of the dyadic product $(v-\mathrm{E}[v])(v-\mathrm{E}[v])^T$.

Since performing this product results in columns (or rows) which are linearly dependent, this term should have rank 1 at most, and so also the expectation of it. Why is this not happening in the case of a variance matrix?

I would like to add an example of what I am trying to say: let us consider a vector random variable with three components: $$v = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix} \mathrm{E}[v]= \begin{pmatrix} \bar v_1 \\ \bar v_2 \\ \bar v_3 \end{pmatrix}$$

the matrix inside the expected value of the variance will be: $$ (v-\mathrm{E}[v])(v-\mathrm{E}[v])^T=\begin{pmatrix} (v_1-\bar v_1)^2 & (v_1-\bar v_1)(v_2-\bar v_2) & (v_3-\bar v_3)(v_1-\bar v_1) \\ (v_1-\bar v_1)(v_2-\bar v_2)& (v_2-\bar v_2)^2 & (v_2-\bar v_2)(v_3-\bar v_3)\\ (v_3-\bar v_3)(v_1-\bar v_1) & (v_2-\bar v_2)(v_3-\bar v_3) & (v_3-\bar v_3)^2 \end{pmatrix} $$ which, without knowing anything about the random variables, should not have full rank, since the second column, for instance, can be written as the first column multiplied by $$\frac{(v_2-\bar v_2)}{(v_1-\bar v_1)}$$ and the determinant is: $$\det(\Sigma_v ) = (v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2 - (v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2 - (v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2 -\left[(v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2-(v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2-(v_1-\bar v_1)^2(v_2-\bar v_2)^2(v_3-\bar v_3)^2\right] = 0$$ I really cannot see where is the bug in my reasoning, but I think it has to be a probabilistic matter... I truly hope someone could help me.

PS:maybe my tags are not perfectly matching the question, sorry.

1

There are 1 best solutions below

3
On

For two singular matrices $A=\left[\begin{array}{ll}1&0\\0&0 \end{array}\right]$ and $B=\left[\begin{array}{ll}0&0\\0&1 \end{array}\right]$, the sum of them is not singular.