What can I learn about the columns with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

59 Views Asked by At

I have a high-dimensional, symmetric matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $d$. By simplifying assumption both L and R have full rank, and the columns of $L$ can be considered low-dimensional projections of the rows of $M$ via the linear map $R$, and vice-versa.

Conceptually, I consider the rows of $M$ to be samples of a vector-valued random variable $m=(m_i)_{1\leq i\leq d}$. Consequently, it makes sense to talk about the (empirical) expectation and variance of each component $m_i$: $Var(m_i)=E[(m_i-E[m_i])^2]$

Ultimately, I'm interested in the image (under the linear projection $\mathbb{R}^{d\times d}\rightarrow \mathbb{R}^{n\times n}$ induced by the matrix decomposition) $\hat{U}\subset\mathbb{R}^{n\times n}$ of a certain subspace $U\subset\mathbb{R}^{d\times d}$. $U$ is spanned by the $k$ unit vectors $e\in\mathbb{R}^{d\times d}$ corresponding to the $k$ dimensions for which $Var(m_i)$ is highest. Analogously, I'm interested in the image of the subspace spanned by the unit vectors corresponding to the dimensions where variance is lowest.

$d$ is very large, and explicitly reconstructing $M$ would be very expensive. Hence my question is: Can I identify a basis of $\hat{U}$ without reconstructing $M$, for example by looking only at the rows of $L$?

Disclaimer: I have also asked this question on CrossValidated, so far with no response.