Finding the closest vector in a subspace to another vector

Question

Let $$W=sp(\{(1,-1,0,0),(1,2,0,-1),(1,0,0,1)\}) \subseteq \mathbb{R}^4$$ Find $w \in W$ closest to $(0,2,1,0)$.

Any hints on how to approach this question?

Answer 1

If $A = \begin{bmatrix}1&1&1\\-1&2&0\\0&0&0\\0&-1&1\end{bmatrix}$, and $\vec{b} = \begin{bmatrix}0\\2\\1\\0\end{bmatrix}$, the system $A\vec{x}=\vec{b}$ has a unique least-squares solution:

$$\vec{x}^* = (A^TA)^{-1}A^T\vec{b}$$

Keep in mind that $\ker A = \{\vec{0}\}$, since the columns of $A$ are linearly independent (by definition of $\operatorname{span}$), so $(A^TA)^{-1}$ is, in fact, computable. Also note that $A\vec{x}=\vec{b}$ has no solutions where $\vec{x}\in W$, since $\vec{b} \not \in W$.

$\vec{x}^*$ is effectively the projection of $\vec{b}$ onto the subspace $W$. It is the minimization of the sum of squares of the components of $\vec{b}-A\vec{x}$, so that $||\vec{b}-A\vec{x}^*|| \le ||\vec{b}-A\vec{x}||, \forall \vec{x} \in \mathbb{R}^4$. This is precisely the vector "closest" to $\vec{b}$.