I have a full-column-rank matrix $A \in \mathbb{R}^{N \times n} $ ($N >> n$):
$Q^{T} A = \begin{bmatrix} R & w \\ 0 & v \\ \end{bmatrix} , Q^{T} = \begin{bmatrix} c \\ d \\ \end{bmatrix} $,
with $R \in \mathbb{R}^{(n-1)\times(n-1)}, w \in \mathbb{R}^{n-1}, v \in \mathbb{R}^{N-n+1}, c \in \mathbb{R}^{n-1}$ and $d \in \mathbb{R}^{N-n+1}$
Now I have to show that
$ \min_x ||Ax - b||^²_2 = ||d||^2_2 - (\frac{v^T d}{||v||_2})^2$
Anyone a idea how to approach this? I don't know how to start with this problem.
What I know is that with a standard least squares problem $\min_x ||y - Fx||_2^2$ with a QR factorization of $F$ and the application of $Q^T$ to $y$
you have the following solution $\hat{x} = R^{-1} d_1$ and $||y-Fx||_2^2 = ||d_2||_2^2$
with $ \begin{bmatrix} Q_1^T \\ Q_2^T \\ \end{bmatrix} y = \begin{bmatrix} d1 \\ d2 \\ \end{bmatrix}$
How to rewrite things to have a format more like this?
$x \in R^n$ must be true, otherwise the matrices cannot be subtracted form each other.
\begin{equation} \begin{split} \displaystyle \min_{x} ||Ax - b||_2^2 = \displaystyle \min_{x} ||Q^T(Ax - b)||_2^2 \\ = \displaystyle \min_{x} ||Q^TAx - Q^Tb||_2^2 \\ = \displaystyle \min_{x} || \begin{pmatrix} Rx & wx \\ 0 & vx \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 \\ \end{split} \end{equation}
This can be written as:
\begin{multline} \displaystyle \min_{x} || \begin{pmatrix} Rx & wx \\ 0 & vx \end{pmatrix} - \begin{pmatrix} 0 & 0 \\ 0 & vx \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 - \displaystyle \min_{x} || \begin{pmatrix} 0 & 0 \\ 0 & vx \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 \\ \end{multline}
Solving the first part of equation 2 following LSQR-Theorom :
\begin{equation} \begin{split} \displaystyle \min_{x} || \begin{pmatrix} Rx & wx \\ 0 & vx \end{pmatrix} - \begin{pmatrix} 0 & 0 \\ 0 & vx \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 \\ = \displaystyle \min_{x} || \begin{pmatrix} Rx & wx \\ 0 & 0 \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 \\ = ||d||_2^2 \end{split} \end{equation}
Than solving the second part of equation 2 following LS-Theorom :
\begin{equation} \begin{split} \displaystyle \min_{x} || \begin{pmatrix} 0 & 0 \\ 0 & vx \end{pmatrix} - \begin{pmatrix} c \\ d \end{pmatrix} ||_2^2 \\ = \displaystyle \min_{x} || vx - d||_2^2 \\ = ((v^Tv)^{-1}v^td)^2 = \left(\frac{v^Td}{||v||_2}\right)^2 \end{split} \end{equation}
Substituting equation 3 and 4 into equation 2 results in the asked answer.
\begin{equation} \displaystyle \min_{x} ||Ax - b||_2^2 = ||d||_2^2 - \left(\frac{v^Td}{||v||_2}\right)^2 \end{equation}
Any chance you are a stundent at TU delft following filtering and identification? Since I had the same exact exercise as homework :). This is what i think works, not the best explanation. It's still fuzzy in my own head. Plus I have doubts about it being $\left(\frac{v^Td}{||v||_2}\right)^2$ because I would expect it being $\frac{v^Td}{||v||_2^2}$. Maybe you have any ideas on that! goodluck and let me know if you think it can be written out better.