Least square solution to a problem of block matrix

185 Views Asked by At

Let $A$ be a $m\times m$ full rank matrix given by $$ A=\begin{bmatrix} R & w \\ 0 & v \end{bmatrix}, \quad R\in \mathbb{R}^{k\times k}, \quad w\in\mathbb{R}^k, \quad v\in\mathbb{R}^{m-k}.$$ If $b$ is a $m\times 1$ matrix given by $$ b=\begin{bmatrix} c \\ d \end{bmatrix},\quad c\in\mathbb{R}^k,\quad d\in\mathbb{R}^{m-k}, $$ I need to prove that $$ \min ||Ax-b||=||d||^2-\left(\frac{v^Td}{||v||}\right)^2. $$

What I did is, considering the normal equation $$A^TAx=A^Tb,$$ we must have $$ \begin{bmatrix} R^TR & R^Tw \\ w^TR & ||w||^2+||v||^2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} R^Tc \\ w^Tc+v^Td \end{bmatrix}, $$ with $x_1\in \mathbb{R}^k,x_2\in\mathbb{R}$. So, $$ \begin{cases} R^TRx_1+R^Twx_2=R^Tc \\ w^TRx_1+(||w||^2+||v||^2)x_2=w^Tc+v^Td. \end{cases} $$

Maybe I could group this equations in a way that uses the normal equations for $R^T, w^T$ and $v^T$, like $$ \begin{cases} R^T(Rx_1+wx_2)=R^Tc \\ w^T(Rx_1+wx_2)+v^T(vx_2)=w^Tc+v^Td, \end{cases} $$ but I don't know if this is useful.

1

There are 1 best solutions below

0
On BEST ANSWER

Here's one approach. Multiplying the first equation (from the left) by $w^TR^{-T}$ and subtracting the result from the second equation yields a result that simplifies to $$ \|v\|^2 x_2 = v^Td \implies x_2 = \frac{v^Td}{v^Tv}. $$ Substituting into the first equation yields $$ R^TR x_1 = - x_2 R^T w + R^Tc \implies\\ Rx_1 = -x_2 w + c \implies \\ x_1 = R^{-1}(c - x_2w). $$ Now, the corresponding vector $Ax$ is given by $$ Ax = \pmatrix{R & w\\0 & v} \pmatrix{x_1\\x_2} = \pmatrix{Rx_1 + wx_2\\ x_2v} = \pmatrix{c-x_2w+x_2w\\x_2v} = \pmatrix{c \\ \frac{v^Td}{v^Tv}v}. $$ It is straightforward to compute $\|Ax - b\|$ from there.