I was reading a recent paper and was trying to understand the novel factor analysis method that they introduce. I am not terrific at linear algebra so I was hoping to get some intuition behind what their method is doing.
Proposed Method
Given two time-series, $X = \{x_t:0≤t<\}$ and $Y = \{y_t:0≤t<\}$, the method first computes the least squares approximation of $Y$ using $X$ by projecting $Y$ onto $X$:
$$ \hat{Y} =YX^\intercal(XX^\intercal)^{-1}X $$
Then, the method performs principal components analysis (PCA) on the resulting least squares approximation $\hat{Y}$. This by done by computing the singular value decomposition of $\hat{Y}$ and keeping the top n singular values:
$$ \hat{Y} = USV^\intercal ≅ U_1S_1{_1}^\intercal $$
Finally, the principal components are extracted by computing:
$$ \hat{Z} = {S_1}^{\frac{1}{2}}{_1}^\intercal $$
I have some questions about this method. First, how exactly can I interpret these principal components? Are they related to the covariance between the two time-series? Also, how would this compare to doing PCA on just one of the time series (e.g. PCA of X)? Finally, how is this related to a linear Gaussian dynamical system -- can this be seen as a linear Gaussian dynamical system where the latent factors Z are used to generate the observed X and Y?
Any insights or thoughts about this method would be much appreciated. Thanks!