I am reading the lecture notes of EE263 of stanford university. I came across these 2 slides (Please see attached images).
I understand every step up to the second slide where it says "residual with optimal x is".
My question is: why is $Ax_{ls} -y = -Q_2 Q^T_2y$ ? I do not know how this is derived. Can someone show me how this is derived ?
It is a little confusing unless you see the trick:
The key here is $\begin{bmatrix} Q_1 Q_2\end{bmatrix} \begin{bmatrix} Q_1^T \\ Q_2^T\end{bmatrix} = Q_1 Q_1^T + Q_2 Q_2^T = I$.
Then $Ax_{ls} - y = Q_1 R_1 x_{ls} - y = Q_1 R_1 R_1^{-1}Q_1^T y - y = (Q_1 Q_1^T-I)y = - Q_2 Q_2^T y$.