Let $L^p$ be the space of all functions with finite p-moment and let $X,Y \in L^p$. Since $L^p$ is a vector space, I thought I could make it an inner product space with $E(XY) : L^p \to \mathbb R$ as the inner product. I have, for $X,Y,Z \in L^2, \lambda\in\mathbb R$:
- $E[(X + Y)Z] = E[XZ + YZ] =E[XZ] + E[YZ] $
- $E[\lambda X] = \lambda E[x]$ both by the linearity of E
- $E[XY] = E[YX]$ is obvious
The definiteness is causing me problems tough. I thought that, since $E[XX] = E[X^2] = \sum_{x\in\mathbb R}x^2p(x)$ (the continuous case is basically the same), with $p(x)$ positive by definition and $x^2$ always positive, we had that $E[X^2]$ is positive for every $X$ and is $0$ if and only if $X$ is almost surely equal to $0$. But then I tried to use this inner product with $X = X-E[X]$ and $Y = Y - E[Y]$ to define the covariance, but I found that $E[(X- E[X])^2] = 0$ for every X almost surely constant, not only almost surely $0$. How can it be, since $x^2$ is always positive? I saw on wikipedia that there is a workaround with a quotient space, but I would like to understand why my approach failed.
EDIT: Basically the questions are:
- Let $X\in L^2$ be a random variable. Then $E[X^2] = \sum_{x\in \mathbb R}x^2p_x(x)$. How can such a sum be $0$ if $X$ is not almost surely equal to $0$?
- Is it possible to define a quotient space of $L^2$ such that $E[XY]$ is also positive definite and thus an inner product? If yes, how?
After quite a bit of discussion, I finally worked out what the issue is, I think. Maybe it's worth recording this as an answer.
Let $(\Omega, P)$ be a probability space. Say two measurable functions (random variables) $X$, $Y$ on $\Omega$ are equal almost surely (a.s) if the set of $\omega \in \Omega$ such $X(\omega) \ne Y(\omega)$ has probability zero. This is an equivalence relation. Moreover, the set $\mathcal N$ of functions $X$ such that $X = 0$ a.s. is a linear space.
There are two possible definitions of $L^2 = L^2(\Omega, P)$:
Let $V^2$ be the linear space of measurable functions $X$ on $\Omega$ such that $E[X^2] < \infty$. This is almost an inner product space, but positive definiteness fails in case $\Omega$ has non-empty subsets $A$ with $P(A) = 0$. Consequently, if one tried to work with $V^2$, one would have a lot of annoyances: convergent sequences with non-unique limits for example.
Let $W^2$ be the set of equivalence classes of functions $X$ in $V^2$ for the equivalence relation of equality a.s. Equivalently, $W^2 = V^2/\mathcal N $. Then $W^2$ is still a linear space and the almost inner product on $V^2$ induces a genuine inner product on $W^2$.
The usual convention in mathematical analysis and in measure theoretic probability theory is to take $W^2$ rather than $V^2$ as $L^2(\Omega, P)$. Nothing is lost by doing this. The only difficulty is that one has to remember that elements of $L^2$ are not actually functions but equivalence classes of functions.
The OP discovered both the difficulty in working with $V^2$ and the solution to the difficulty, which is to replace $V^2$ with $W^2$.