Uniqueness of orthogonal projection in closed subspace of Hilbert space

Question

Let $A\subseteq H$ be a closed subspace of a Hilbert space $H$. Show that for each $f\in H$, there exists a unique element $a\in A$ so that $(a - f)\perp A$ (i.e. $a-f\in A^\perp$).

By definition, a Hilbert space is a complete inner product space. Since $A$ is closed in $H$, it is also complete. Also, one can choose $y_n$ so that $\lim\limits_{n\to\infty} \lVert f - y_n\rVert = \mathrm{dist} (f,A) := \inf\{\lVert f- a\rVert : a \in A\}.$ Then one could let $a = \lim\limits_n y_n,$ assuming it exists (to show it's in $A$ it suffices to show that the sequence is Cauchy). Why does it exist?

If this holds, I think I can show that $b\in A, \langle a - f,b\rangle = 0,$ but I'm not sure how.

Answer 1

Step 1. Denote $d=\mathrm{dist}(f,A) := \inf\{\lVert f- a\rVert : a \in A\}.$ For each $n\in\mathbb N$, we can take $y_n\in A$ such that $\|f-y_n\|<d+\frac1n$. We show that $\{y_n\}$ is Cauchy.

We use Parallelogram identity to compute $\|y_n-y_m\|^2$. For each $n,m\in\mathbb N$, we have \begin{align*} \|y_n-y_m\|^2&=\|(f-y_n)-(f-y_m)\|^2\\ &=2\left(\|f-y_n\|^2+\|f-y_m\|^2\right)-4\left\|f-\frac{y_n+y_m}2\right\|^2\\ &\leq 2\left(\left(d+\frac1n\right)^2+\left(d+\frac1m\right)^2\right)-4d^2\\ &=4\left(\frac1n+\frac1m\right)d+2\left(\frac1{n^2}+\frac1{m^2}\right). \end{align*} From this we can easily show that $\{y_n\}$ is Cauchy.

Since $A$ is closed, the limit $a=\lim y_n$ belongs to $A$. We also have $\|f-a\|=d$.

Step 2. Now we prove that $\langle a - f,b\rangle = 0$ for any $b\in A$. We may assume WLOG that $b\neq 0$. For any $t\in\mathbb R$, we have $tb+a\in A$, hence $\|a-f+tb\|^2\geq d^2=\|a-f\|^2$ for all $t$, i.e., (Here I assume that the Hilbert space is over $\mathbb R$. If $H$ is a complex Hilbert space, this argument needs to be slightly modified.) $$t^2\|b\|^2+2\langle a - f,b\rangle t+\|a-f\|^2\geq\|a-f\|^2,\qquad\forall t\in\mathbb R.$$ Or equivalently, $$t^2\|b\|^2+2\langle a - f,b\rangle t\geq 0,\qquad\forall t\in\mathbb R.$$ This implies that $\langle a - f,b\rangle = 0$ by taking $t=-\frac{\langle a-f,b\rangle}{\|b\|^2}$. Indeed, this $t$ also can be used for complex Hilbert space $H$! If $H$ is complex and $t=-\frac{\langle a-f,b\rangle}{\|b\|^2}\in\mathbb C$, then we have \begin{align*} \|a-f\|^2\leq \|a-f+tb\|^2&=\|a-f\|^2+\bar t\langle a-f,b\rangle+t\overline{\langle a-f,b\rangle}+|t|^2\|b\|^2\\ &=\|a-f\|^2-\frac{|\langle a-f,b\rangle|^2}{\|b\|^2}, \end{align*} which gives that $\langle a-f,b\rangle=0$.

Step 3. Uniqueness. If $a, a_1\in A$ satisfy $(f-a)\perp A$ and $(f-a_1)\perp A$, then $a-a_1=(f-a_1)-(f-a)\in A^\perp$. That is, $\langle a-a_1, b\rangle =0$ for all $b\in A$. Taking $b=a-a_1\in A$ gives that $\|a-a_1\|=0$ and thus the uniqueness.

Answer 2

The parallelogram law is useful in this regard: $$ \|x-y\|^2+\|x+y\|^2 = 2\|x\|^2+2\|y\|^2. $$ For $z\notin A$, let $\{ x_n \}_{n=1}^{\infty}$ be a sequence of vectors in $A$ chosen so that $$ \|z-x_n\| \le \mbox{dist}(z,A)+\frac{1}{n} \tag{*} $$ Applying the parallelogram law to $x=(z-x_n),y=(z-x_m)$ gives $$ \|x_n-x_m\|^2+4\|z-\frac{1}{2}(x_n+x_m)\|^2 =2\|z-x_n\|^2+2\|z-x_m\|^2 $$ Therefore, $$ \|x_n-x_m\|^2+4\cdot\mbox{dist}(z,A)^2 \\ \le 2\left(\mbox{dist}(z,A)+\frac{1}{n}\right)^2+2\left(\mbox{dist}(z,A)+\frac{1}{m}\right)^2 \\ = 4\cdot\mbox{dist}(z,A)^2+4\left(\frac{1}{n}+\frac{1}{m}\right)\mbox{dist}(z,A)+2\left(\frac{1}{n^2}+\frac{1}{m^2}\right) $$ This gives $$ \|x_n-x_m\|^2 \le 4\left(\frac{1}{n}+\frac{1}{m}\right)\mbox{dist}(z,A)+2\left(\frac{1}{n^2}+\frac{1}{m^2}\right). $$ Therefore $\{ x_n \}_{n=1}^{\infty}$ is a Cauchy sequence of vectors in $A$, which must converge to some $a\in A$ because $\mathcal{H}$ is complete and $A$ is closed. Furthermore $\|z-a\| \le \mbox{dist}(z,A) \le \|z-a\|$, which, by equation $(*)$ gives $$ \|z-a\|=\mbox{dist}(z,A). $$ Therefore, there exists a closest point projection of $z$ onto $A$. Furthermore $a\in A$ is unique in this regard because, if $a'\in A$ satisfies $\|z-a\|=\|z-a'\|=\mbox{dist}(z,A)$, then $a=a'$ because of the parallelogram law; indeed $$ \|(z-a)+(z-a')\|^2+\|(z-a)-(z-a')\|^2 \\ = 2\|z-a\|^2+2\|z-a'\|^2 $$ implies $$ 4\|z-\frac{1}{2}(a+a')\|^2+\|a-a'\|^2=4\mbox{dist}(z,A)^2 \\ 4\mbox{dist}(z,A)^2+\|a-a'\|^2 \le 4\mbox{dist}(z,A)^2 \\ \implies a=a'. $$ Therefore, all closest point projections to $A$ are the same. I'll leave it to you to show that a closest point projection $x_p$ of $x$ onto $A$ is the orthogonal projection of $x$ onto $A$.