Let $X$ be a Hilbert space, with $A$ and $B$ closed linear subspaces of $X$ and $A\subsetneq B$.
- How do I prove that $A^\perp\cap B\neq \{0\}$?
- Does this hold if $A$ is not closed?
Unfortunately I have no idea where to start this proof, so I would greatly appreciate any push in the right direction.
Also for 2., I would say that it does not, but which counter-example could I give?
For closed subspaces of Hilbert spaces you have a map that is called the orthogonal projection onto the subspace. This is a hermitian map from $H$ to $A$ that restricts to the identity on $A$.
Let $\pi_A : H\to A$ be the projection onto $A$, $v\in B-A$. Then $v-\pi_A(v)$ lies in $B$ since $v$ and $\pi(v)$ lie in $B$. It cannot be zero, since $v\notin A$. But for every $w\in A$ you have that $$\langle w, v-\pi(v)\rangle=\langle \pi(w),v-\pi(v)\rangle=\langle w,\pi(v)-\pi^2(v)\rangle = \langle w, 0\rangle=0$$
So $v-\pi(v)$ also lies in $A^\perp$.
For the second part it does not need to hold if $A$ is not closed. As an example consider $H=B$ and $A$ a dense but not closed subspace of $H$. Since $A$ is dense for every $v\in H$ you have a sequence $v_n\to v$ with $v_n\in A$. Let now $w\in A^\perp$ and $v\in H$ arbitrary, then $$\langle w,v\rangle =\lim_n \langle w,v_n\rangle=0$$ from continuity of the scalar product. Since this holds for every $v\in H$ you have $w=0$ and thus $A^\perp =\{0\}$.
But it is true if the closure of $A$ lies in $B$.