I am doing an exercise created by myself :). Let $A \in \mathbb{R}^{n \times p}, C \in \mathbb{R}^{m \times p}, B \in \mathbb{R}^{n \times q}, E \in \mathbb{R}^{m \times q}$ be any real matrices and Let $$M_{1}=A^{\top} A+C^{\top} C$$ $$ M_{2}=\left(A^{\top} B+C^{\top} E\right)\left(E^{\top} E+B^{\top} B+I_{q}\right)^{-1}\left(B^{\top} A+E^{\top} C\right) $$ obviously $M_{1}$ and $M_{2}$ are positive semidefinite matrix, prove that $M_{1}-M_{2}$ is also positive semidefinite matrix, i.e minimum eigenvolue of $M_{1}-M_{2}$ bigger than 0 .
My approach: I try SVD and expand $M_{2}$ but I don't know how to analyze the cross-product term. I run many simulations in which the entries of $A,B,C,E$ are from i.i.d Gaussian. In all simulations the minimum eigen-value of $M_1-M_2$ is greater than zero.(I set p=q=n=10,m=300)
I wonder if someone can help me prove this or give me an counter example on above statement.
Oh, I think I prove it, my approach is that: Let $$W=\left(\begin{array}{c} B \\ E \end{array}\right),Q=\left(\begin{array}{c} A \\ C \end{array}\right)$$ Then $M_1=Q^TQ,\quad M_2=Q^TW(W^TW+I_q)^{-1}W^TQ \Rightarrow M_1\succ M_2$
Your approach works, but your have yet to explain why $Q^TQ\succeq Q^TW(W^TW+I_q)^{-1}W^TQ$. Observe that $W(W^TW+I_q)^{-1}W^T$ share the same set of nonzero eigenvalues with $W^TW(W^TW+I_q)^{-1}=I_q-(W^TW+I_q)^{-1}$. Hence all eigenvalues of $W(W^TW+I_q)^{-1}W^T$ are between $0$, and $1$ and $I_{n+m}\succeq W(W^TW+I_q)^{-1}W^T$. In turn, $Q^TQ\succeq Q^TW(W^TW+I_q)^{-1}W^TQ$.
An easier approach is to note that $M_1-M_2$ is the Schur complement of the bottom right diagonal sub-block of $$ P=\pmatrix{A^T&C^T\\ B^T&E^T}\pmatrix{A&B\\ C&E}+\pmatrix{0&0\\ 0&I_q}. $$ Since $P$ is positive semidefinite, the Schur complement $M_1-M_2$ in $P$ must also be positive semidefinite.