So, I've been learning linear algebra from a rather old book I really like (Andrzej Białynicki-Birula, "Geometria z algebrą liniową") that talks a bit about "generalized inner products" (that is, bilinear symmetric functionals $\xi$ where $\xi(x, x)$ might equal anything) and this problem comes across as a little strange:
Let $(V, \xi)$ be a linear space $V$ over a field $K$ with such a bilinear functional $\xi:V \times V \to K$. Then the set $S = \{x \in V : \xi(x, x) = 0\}$ is linear subspace of $V$ iff either of [REDACTED] conditions: $\mathrm{char}\,K = 2$ or $S \subseteq V^\perp$ ($V^\perp$ being set of all $x \in V$ such that for all $y \in V$, $\xi(x, y) = 0$) hold.
What the hell? That is such a blatant lie, after all for Euclidean spaces $S$ would be null space, or if $\xi = 0$ then this obviously doesn't hold. The second condition makes intuitive sense: if it holds, then using bilinearity $S$ is clearly a linear subspace, but I can't seem to get anything going trying to prove the inverse. I'm just wondering what is the actual connection between these concepts the author might have intended to capture.
Thanks!
EDIT: Got it, the problem mistakenly said the two conditions are equivalent. I redacted it.
Caveat: this is a partial answer as I don't have time to fill every detail right now.
You want to think of $S$ as the lightcone of $(V,\xi)$, and we'll assume that $\xi$ is symmetric. Your attempted counter example for the Euclidean space does not work, because $S = \{0\}$ is a subspace of $\Bbb R^n$, and $\{0\} \subseteq (\Bbb R^n)^\perp = \{0\}$. There is nothing wrong with this.
If ${\rm char}(K) = 2$, then $x,y \in S$ implies that $$\xi(x+y,x+y) = \xi(x,x) + 2\xi(x,y)+\xi(y,y) = 0+0+0 = 0,$$so $S$ is a subspace of $V$. The first term dies because $x \in S$, the second one because ${\rm char}(K) = 2$, and the third because $y \in S$.
If $S \subseteq V^\perp$, then $x,y \in S$ implies that $$\xi(x+y,x+y) = \xi(x,x) + 2\xi(x,y)+\xi(y,y) = 0+0+0 = 0,$$so $S$ is a subspace of $V$. The first term dies because $x \in S$, the second one because $S \subseteq V^\perp$, and the third because $y \in S$.
For the converse, assume that $S$ is a subspace. If $S \subseteq V^\perp$, there is nothing to do. So assume that there is $x \in S$ that is not in $V^\perp$ and let's show that ${\rm char}(K)=2$. Since $x$ is not in $V^\perp$, the linear functional $\xi(x,\cdot): V \to K$ is non-zero, hence surjective. We will be done once we show that the restriction $\xi(x,\cdot)|_S$ is also non-zero (hence also surjective), since for every $\lambda \in K$ there will be $y_\lambda \in S$ with $\xi(x,y_\lambda) = \lambda$, leading to $0 = \xi(x+y_\lambda,x+y_\lambda) = 2\lambda$ as wanted.
I could not find a fast way to argue that $\xi(x,\cdot)|_S \neq 0$, though.