Understanding the definition of method of least squares

45 Views Asked by At

I'm a little confused by the definition of the method least squares. This is the definition I'm looking at

Let $W$ be a finite dimensional subspace of an inner product space $V$. If $\vec{v} \in V$, then the vector closest to $\vec{v}$ in $W$ is $proj_{w}\vec{v}$. That is $$ ||\vec{v}-proj_{w}\vec{v}|| < ||\vec{v}-\vec{w}|| $$ for all $\vec{w} \in W$ and $\vec{w} \neq proj_{w}\vec{v}$

What is $\vec{w}$ in this definition and why do we need the condition that $\vec{w} \neq proj_{w}\vec{v}$

1

There are 1 best solutions below

0
On BEST ANSWER

Consider a function $f$ that maps each element $\vec{w}$ of $W$ to the distance $|| \vec{v} - \vec{w} ||$. Given that $f(\vec{w})$ is non-negative for every $\vec{w} \in W$, it is not unreasonable to expect that there exist one or more vectors that minimize $f$. By that, I mean that a vector $\vec{w}_{0} \in W$ is a minimum for $f$ if for any $\vec{w} \in W$, we have $$ f(\vec{w}_{0}) \leq f(\vec{w}). $$ In fact, $f(\vec{w}_{0}) = f(\vec{w})$ if and only if $\vec{w}$ is another minimum of the function. Therefore, the statement that $$ f(\vec{w}_{0}) < f(\vec{w}) $$ for all $\vec{w} \not = \vec{w}_{0}$ means that $\vec{w}_{0}$ is the unique minimum of $f$.

Thus the definition can be restated as saying that the projection of $\vec{v}$ onto $W$ is the unique vector $\vec{w}$ in $W$ that minimizes the function $f(\vec{w}) = || \vec{v} - \vec{w} ||$.