Using upper-triangular matrix and orthogonal matrix to solve least square problem

112 Views Asked by At

Through transformation with orthogonal matrix $O$, the problem $\hat{b}=\operatorname{argmin}_b \| y-Xb \|^2$ is equivalent to $\hat{b}=\operatorname{argmin}_b \| y^*-X^*b \|^2$ where $y$ and $y^*$ are in $\textbf{R}^m$, $X$ and $X^*$ are in $\textbf{R}^{m \times n}$ $(m\geq n)$, and $y^*=Oy$ and $X^*=OX$. Let $y^*=[y_1^*,y_2^*,\dots,y_m^*]^t$. If $X$ is upper-triangular, prove that the residual sum of square $$\| y-X\hat{b} \|^2 = \sum_{i=n+1}^m > |y_i^*|^2$$

In my attempts, I don't know why $y_1^*,\dots,y_n^*$ are not included in the right side of the equation, since $X^*$ is an upper-triangular matrix.