We know that:
$$\forall x,y\in V, |\langle x,y\rangle|^2 \leq |\langle x,x\rangle||\langle y,y\rangle|$$
And in statistics we have for all $\zeta,\eta$ being random variables,
$E(\zeta \eta) \le \sqrt{E\zeta^2E\eta^2}$
Now, clearly there is some relation here which I'm trying to figure out. Are $\zeta, \eta$ something like n-dimensional vectors and is the expected value E something like an inner product or a norm?
On
The former is a special case of the latter.
Given two random variables $X$ and $Y$ we can define their inner product as
$$\langle X,Y \rangle = E(XY)$$
See https://en.wikipedia.org/wiki/CauchySchwarz_inequality#Probability_theory for more.
The expectation of a random variable $X: \Omega \to \mathbb R$ where $\Omega = (\Omega, \mathcal{F}, P)$ is a probability space, is defined as:
$$\mathbb E(X) = \int_\Omega X dP$$
In general, if $(X,A, \mu)$ is a measure space and $f_1,f_2: X \to \mathbb R$ are measurable functions, then
$$\left( \int_X |f_1 f_2| d\mu \right)^2 \le \left( \int_X |f_1|^2 d\mu \right)\left( \int_X|f_2|^2 d\mu \right)$$
this follows from the more general Holder's inequality, or, even better for us here, if the functions are assumed to be square integrable, this follows from the fact that $L^2(\Omega)$ is a Hilbert space, so it is a direct consequence of the usual Cauchy-Schwarz in an inner product space. It's usually called the Cauchy-Schwarz inequality.
In the case of random variables, we have:
$$\left( \mathbb E[|XY|] \right)^2 = \left( \int_{\Omega} |XY| dP \right)^2 \le \left( \int_{\Omega}|X|^2 d P \right) \left(\int_{\Omega} |Y|^2 dP\right) = \mathbb E[X] \mathbb E[Y]$$