the meaning of the term "dimension" in the context of Hilbert space

Question

Hi: This is a novice Hilbert space question but it's always where I've gotten stuck when reading various books-documents on Hilbert space. I think an example will clarify my confusion.

I'm currently reading the umpteenth explanation of Hilbert spaces. After some intro material, there is a new section and the title of the section is "A Hilbert Space of Random Variables". The section starts with the following statement:

Let $R_{0}$ be the vector space of zero-mean random variables with finite second moments defined on a common probably space $(\Omega, F, P)$ endowed with the inner product $\langle X, Y\rangle = E(X Y)$, norm $\|X \| = \sqrt{E(X^2)}$ and metric $\|X - Y\|$.

They then prove that the space $R_{0}$ is a Hilbert Space.

This is where I always get stuck and look for another explanation but I never found anything that clears up the following confusion.

It is usually emphasized in everything that I read about Hilbert spaces that one way to think about Hilbert space is that it is an infinite dimensional space analogous to $R^{n}$ except that $n \rightarrow \infty$.

But, in the case of the vector space of zero mean univariate random variables in the example at the beginning of the section, to me, from a linear algebra perspective, the dimension of $X$ is one. In fact, it has to be or they would not be able to write $E(X^2)$.

So, it seems to be the case that, when the term "infinite" dimensional is used, they are not referring to the dimension of the vector (i.e: object) in the space but rather than the number of vectors (i.e: objects) that are needed when in order to complete the space. By this I mean that you need an linear combination of an infinite number of the vectors in order to generate all the other vectors in the space. But the vectors themselves are NOT infinite dimensional.

Is that correct, namely that "infinite" dimensional is NOT referring to the actual dimension of the objects in the space. So, for example, you can have a Hilbert space that is an infinite set of vectors in $R^{n}$ so that each vector actually has dimension $n$ but there are an infinite number of them. Thanks for confirmation or correction. I'd like to be finally able to move on in my Hilbert space reading and get past this blockade.

Answer 1

The dimension refers to the size (in a sense) of the set of vectors, not to the size or dimension (in any sense) of the individual vectors.

The space of real $n$-tuples is $n$-dimensional because every $n$-tuple is a linear combination of the $n$ standard unit coordinate vectors.

The space of random variables you ask about has an infinite basis: a countable set of random variables such that every random variable is a linear combination of them - allowing for infinite sums that converge.

It's not the limit of $\mathbb{R}^n$ as $n$ grows.

You should think of $E(X^2)$ as the square of the length of the vector $X$.

Answer 2

A random variable is a $P$-measurable function on $Ω$. Depending on the situation, the probability space $Ω$ can already be very large. For example, if you consider events that are influenced by a Brownian motion/Wiener process, then $Ω$ is (or is at least as large as) the space of continuous functions.

The Hilbert space you consider is a subset of $L^2(Ω,P)$. For every collection of disjoint measurable subsets of $Ω$, that is, elements of $\cal F$, the functions that are constant on these subsets and zero everywhere are random variables. If $Ω$ is not finite (and $\cal F$ not too strange), then this way you can construct subspaces of every dimension of the given Hilbert space. Thus you get infinite dimension for the Hilbert space itself.

There are nice situations where the Hilbert space has a countable topological basis (the space is the closure of the linear hull). If $Ω$ is finite dimensional and $\cal F$ based on the Borel algebra, you get that. All the examples where approximations (of finite data size) can be computed fall into this class. But in general the space can be much larger.