Let $H$ be a real Hilbert space with inner product $(\cdot, \cdot)$. Assume that there is a subset $C \subset H$ that satisfies
$$C=\{x \in H : (x,y) \geq 0\} \forall y \in C$$
$C$ can be shown to be a non-empty convex cone, and I have already verified this for myself. Define $P_C(x)$ to be the orthogonal projection of $x$ onto $C$ so that
$$||P_C(x)-x||_H\leq ||y-x||_H \forall y \in H$$
My Question:
Show that $(P_C(x),P_C(x)-x)=0$
Using properties of inner products did not get me anywhere, so my next thought was to verify the Pythagorean identity for these two vectors. However, this did not amount to anything after expansion of $||P_C(x)+P_C(x)-x||^2.$ A hint on how to show this would be excellent. I know how to show this result in $\mathbb{R}^n$ where the orthogonal projection of $v$ onto $u$ is defined $\frac{v \cdot u}{u \cdot u}u,$ I am just trying to prove this generalization.
It should also be noted that $C \cap -C = \{0\},$ which I have also proven.
NOTE: This is not homework, just something I encountered in literature that I would like to verify. Thank you for the help.
Your question more generally is to show the orthogonality result for any (possibly nonconvex) set that is a cone.
Let $H$ be a real Hilbert space. Let $D\subseteq H$ be any (possibly nonconvex) set with the property that if $d \in D$ then $ad \in D$ for all $a\geq 0$.
Claim: Fix $x \in H$ and let $p\in D$ be a point that satisfies $$ ||d-x||^2 \geq ||p-x||^2 \quad \forall d \in D$$ Then
a) We have $$ ||d-p||^2 \geq 2\langle d-p,x-p\rangle \quad \forall d \in D$$
b) $\langle p, p-x\rangle = 0$.
Proof:
To prove (a), fix $d\in D$. Then $$ ||d-x||^2 \geq ||p-x||^2$$ and so $$ ||d-p+p-x||^2 \geq ||p-x||^2$$ that is $$ ||d-p||^2 + 2\langle d-p, p-x\rangle + ||p-x||^2\geq ||p-x||^2$$ Thus $$ ||d-p||^2 + 2\langle d-p, p-x\rangle \geq 0$$ Rearranging terms yields (a).
To prove (b), observe that since $p \in D$, we know for all $\epsilon \in (0,1)$ we have $(1-\epsilon)p \in D$ and $(1+\epsilon)p \in D$. It follows by part (a) that for all $\epsilon \in (0,1)$:
$$||(1-\epsilon)p - p||^2 \geq 2\langle (1-\epsilon)p-p, x-p\rangle \quad (*)$$ $$ || (1+\epsilon)p - p||^2 \geq 2\langle (1+\epsilon)p-p,x-p\rangle \quad (**)$$
Simplifying gives: $$(*)\implies \epsilon^2||p||^2 \geq -2\epsilon \langle p, x-p\rangle$$ $$(**)\implies \epsilon^2||p||^2 \geq 2\epsilon \langle p, x-p\rangle$$ Thus $$ \epsilon^2 ||p||^2 \geq 2\epsilon |\langle p, x-p\rangle| \quad \forall \epsilon \in (0,1)$$ Dividing by $\epsilon$ gives $$ \epsilon ||p||^2 \geq 2 |\langle p, x-p\rangle| \quad \forall \epsilon \in (0,1)$$ Taking a limit as $\epsilon \rightarrow 0$ gives $$ 0 \geq 2 |\langle p, x-p\rangle|$$ and so $\langle p, x-p\rangle = 0$. $\Box$
Since the set $D$ is not required to be convex, there may be more than one point $p \in D$ that is closest to $x$. So, the above claim holds for all such closest points $p$. (It just assumes there exists at least one such closest point.) An example is $H=\mathbb{R}^2$ and $$D=\{(a,0) : a\geq 0\}\cup\{(0,b):b\geq 0\}$$ Let $x=(1,1)$. Then $p_1=(0,1)$ and $p_2=(1,0)$ are two distinct closest points in $D$ to $x$, and indeed we have $\langle p_1,p_1-x\rangle=0$ and $\langle p_2, p_2-x\rangle=0$.