When i proved that:
Let $(H,\|\cdot\|_q)$ be a Hilbert Space (over $\mathbb{R}$). Let $w_1\in W<H$ ($W$ is a subspace of $H$) and $v\in H$, such that $v-w_1\in W^\perp$. Prove that $w_1$ is the best approximation of $W$ to $v$ , i.e., prove that $\|w_1-v\|_q\leq \|w-v\|_q \quad \forall w\in W$
I came up with the question: Is the converse also valid?, i.e. If $w_1$ is the best approximation to $v$ of $W$, is then $v-w_1 \in W^\perp$?
First, I thought very specific, If I had an orthogonal set, say: $\lbrace u_1,u_2,\ldots,u_n\rbrace \subset H$, then I know that the best approximation to $v\in H$ of $W$ is $\sum_{i=1}^n q(v,u_i)u_i$ and $v-\sum_{i=1}^n q(v,u_i)u_i\in W^\perp$. But, is this true in general?, i.e. that for every Hilbert Space, and given the $v\in H$, and $W<H$, can we construct an orthogonal set such that the sum (similar to the sum above) is in $W^\perp$?
Thanks in advance.
Yes. The standard argument goes like this.
Suppose $w_1$ is the best approximation. Let $w \in W$ be arbitrary. Then for any $s \in \mathbb{R}$, we must have $\|w_1 -v\| \le \| (w_1 + sw) - v\|$. So if we set $\gamma(s) = \|w_1 + sw - v\|^2$, then $\gamma$ attains its minimum at $s=0$. Now expanding, $$\gamma(s) = \|w_1 - v\|^2 + 2 s \langle w_1 - v,w\rangle + s^2 \|w\|^2$$ from which it is clear that $\gamma$ is smooth in $s$ and its derivative is $$\gamma'(s) = 2 \langle w_1 - v,w \rangle + 2 s \|w\|^2.$$ Since it attains a minimum at $s=0$, we must have $$0 = \gamma'(0) = 2 \langle w_1 - v,w \rangle$$ so that $w_1 - v$ is orthogonal to $w$. But $w \in W$ was arbitrary.