Prove that if $u$ is a least squares solution of $Bx = b$ then $u$ $=$ $R^{-1}$ ($Q^T$) $b$.

Question

Let $B$ be an $m \times n$ matrix, whose columns are linearly independent. Suppose that $B$ has a $QR$ factorization, i.e., $B = QR$ where $Q$ is an $m \times n$ matrix with orthonormal columns and $R$ is an $n \times n$ upper triangular matrix with positive entries along its diagonal. Show that if $u$ is a least squares solution of $Bx = b$ then $u$ $=$ $R^{-1}$$Q^T$ $b$.

Answer 1

Step 1: The least squares solution is unique.
Note that $\mathrm{Nul}\,B^T B = \mathrm{Nul}\,B$. Since the columns of $B$ are linearly independent, $\mathrm{Nul}\,B=\{0\}$ and hence $\mathrm{Nul}\,B^T B = \{0\}$. Since $B^T B$ is square, it is invertible. Hence there is a unique solution to the least squares equation $B^T B x = B^T b$, namely $x= {(B^T B)}^{-1}B^T b$.

Step 2: $u = R^{-1}Q b$ is the least squares solution.
Fill in $B=QR$ in the expression $x={(B^T B)}^{-1}B^T b$ and you get $$ \begin{align*} x &= {(R^T Q^T Q R)}^{-1}(Q R)^T b \\ &= {(R^T R)}^{-1} R^T Q^T b \\ &= R^{-1}{(R^T)}^{-1}R^T Q^T b \\ &= R^{-1}Q^T b. \end{align*} $$