Singular value of production of projection matrix and some other matrix

Question

if I have a matrix $X\in\mathbb{R}^{n\times d}$, and some projection matrix $P\in\mathbb{R}^{n\times n}$ which projects things onto a $k$-dim subspace of the column space of $X$, is there some expression for the singular values of $PX$? Say WLOG that $\|X\|_{op}\leq1$. Using Weyl's inequality we get an upperbound that $\sigma_i(PX)\leq \sigma_i(X)$. But I suspect that this is too pessimistic since $P$ projects onto a subspace of $col(X)$. For example, if $PX=X_k$ being the SVD of the top $k$ singular values, then $\sigma_1(PX)\leq \sigma_{k+1}(X)$ which can be very small. Thanks for any advice!

Answer 1

This is a bit of reasoning, but not a final answer.

Lemma 1: Let $U\in\mathbb R^{n\times n}$ be a unitary matrix and $P\in\mathbb R^{n\times n}$ be a projection matrix, then $PU=UP'$ with $P'$ a projection with same rank as $P$.

Proof : Write $P=\sum_{i=1}^k p_i\cdot p_i^T$ with $\{p_i\}_{i=1}^k$ an othonormal set of vectors. Then \begin{align*} PU&=\sum_{i=1}^k p_i\cdot (U^T p_i)^T\\ &=U\sum_{i=1}^k (U^T p_i)\cdot (U^T p_i)^T\\ &=U P' \end{align*}

It is clear from that, that we can essentially assume without loss of generality that $X$ is diagonal with positive values, indeed if $X=U \Sigma V^T$ then the question doesn't depend on $V$ and $PX=UP'\Sigma V$ where the relation between $P$ and $X$ translates equivalently to the identical relation on $\Sigma$ and $P'$. Note that the relation on $\Sigma$ and $P'$ is essentially just that $P'=\begin{bmatrix} P''&0\\0&0 \end{bmatrix}$ and $\Sigma=\begin{bmatrix} \Sigma'&0\\0&0\end{bmatrix}$ with similar shapes (the top left matrices are $p\times p$), therefore we can assume WLOG that $\Sigma$ is full rank and that $P''$ is any projection matrix. The question therefore becomes

If $\Sigma$ is a full rank diagonal matrix with positive values and $P$ a projection matrix, then what are the singular values of $P\Sigma$.

Let's have some examples to browse for intuition. Write $\Sigma=\begin{bmatrix} \sigma_1&0&\cdots&0\\0&\sigma_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\sigma_n \end{bmatrix}$ with $\sigma_1\geq \sigma_2\geq\dots\geq \sigma_n>0$.

Example 1: Let $P=\begin{bmatrix} I_k&0\\0&0 \end{bmatrix}$, then the singular values of $P\Sigma$ are $\sigma_1,\sigma_2,\dots, \sigma_k$.

Example 2: Let $P=\begin{bmatrix} 0&0\\0&I_k \end{bmatrix}$, then the singular values of $P\Sigma$ are $\sigma_{n-k+1},\sigma_{n-k+2},\dots, \sigma_n$.

Example 3: Suppose $n=2$, $k=1$ and $\sigma_1 > \sigma_2$, then $P=uu^T$ for some unit $u=\in\mathbb R^2$. Observe that $u$ is a left singular vector with singular value $\sqrt{u^T \Sigma^2 u}$, indeed if $v=(P\Sigma)^T u=\Sigma u$ and $P\Sigma v = P\Sigma^2 u=u \cdot (u^T \Sigma^2 u)$.

The last example might be a bit misleading since it doesn't generalize to higher dimensions, it is not true that if $P=\sum_{p=1}^k u_i u_i^T$ then $u_i$ is a left singular vector of $P\Sigma$.

I can't quite finish the argument but I think this still gives a nice characterization of the extreme scenarios and a way to dive into a full characterization of all possible achievable sets of singular values for $P\Sigma$ and therefore for $PX$.