Inequality of vector norms with projections

Question

Let $W$ be a vector subspace of $V$, a space with a dot product; $v\in V$. Let $p_W(v)$ be the orthogonal projection of $v$ onto $W$ and $w\in W, w\neq p_W(v)$.

How can i prove that $||v-w|| > ||v-p_W(v)||$?

Answer 1

Let $v = v_\| + v_\bot $, where $v_\| = p_W(v)$, $v_\bot \in W^\bot$.

$ \|v-w\|^2 = (v-w)^2 = v^2 + w^2 - 2vw = v_\bot^2 + v_\|^2 + w^2 - 2v_\|w = v_\bot^2 + (v_\|-w)^2$

$ \|v-v_\|\|^2 = v_\bot^2 < v_\bot^2 + (v_\|-w)^2 $, QED.

Answer 2

Since $W \subset V$ is a fixed subspace, I am going to drop the subscript "$W"$ from $P$; in this answer, $P = P_W$.

Recall that an orthogonal projection $P$ satisfies

$P^2 = P = P^T; \tag 1$

thus for any

$x, y \in V \tag 2$

we have

$\langle x, Py \rangle = \langle P^Tx, y \rangle = \langle Px, y \rangle; \tag 3$

now consider the three vectors $v$, $Pv$, and $w$;

$Pv, w \in W \Longrightarrow Pv - w \in W, \tag 4$

$\langle v - Pv, w \rangle = \langle v, w \rangle - \langle Pv, w \rangle = \langle v, w \rangle - \langle v, Pw \rangle = \langle v, w \rangle - \langle v, w \rangle = 0, \tag 5$

where we have used (3) and the fact that $Pw = w$ for $w \in W$; since this holds for every $w \in W$, we have established that

$v - Pv \in W^\bot; \tag 6$

we write

$v - w = (v - Pv) + (Pv - w), \tag 7$

and by virtue of (4) and (6)

$\langle v - Pv, Pv - w \rangle = 0, \tag 8$

whence

$\Vert v - w \Vert^2 = \langle v - w, v - w \rangle = \langle (v - Pv) + (Pv - w), (v - Pv) + (Pv - w) \rangle$ $= \langle v - Pv, v - Pv \rangle -2\langle v - Pv, Pv - w \rangle + \langle Pv - w, Pv - w \rangle$ $= \Vert v - Pv \Vert^2 + \Vert Pv - w \Vert^2;\tag 9$

since each term on the right is non-negative, we find that

$\Vert v - w \Vert^2 \ge \Vert v - Pv \Vert^2, \tag{10}$

whence

$\Vert v - w \Vert \ge \Vert v - Pv \Vert, \; \forall w \in W, \tag{11}$

with equality holding precisely when

$\Vert w - Pv \Vert = 0 \Longleftrightarrow w = Pv; \tag{12}$

thus

$w \ne Pv \Longrightarrow \Vert v - w \Vert > \Vert v - Pv \Vert. \tag{13}$

$OE\Delta$.

Inequality of vector norms with projections