What does the Cauchy Criterion entail in a Hilbert space?

Question

I'm taking an introductory course in Machine Learning and we are looking at all the requirements a Vector Space must fufill to be a Hilbert Space. I'm not a very math versed person, i had to recently learn the inner product, equivalence relations, and what it even meant to use functions as vectors. I think those concepts are fine for me now but the Cauchy Criterion is where I feel completly lost. I have watched some Youtube videos on the matter, they usued a number line from 0 to 1 and points in between from which the distance in between to consecuitive points had to be less then a value epsilon. While the Cauchy Criterion made sense in such context I fail to see how it applies to a vector space of functions,discrete functions or random valued functions.

I though that the vectors that I would encounter in a Hilbert space would be functions like sinusoids or some signal's that decay and are convergent. Are all vectors on a Hilbert Space then part of a Cauchy sequence? Can, for example, sin(x) not be part of a Hilbert space because it's not part of a Cauchy sequence? Can there be elements of a cauchy sequence and other elements that are not part of a cauchy sequence? What if there are no other elements of a cauchy sequence in my vector space and just normal linear algebra vectors, if I use a Hilbert Space inner product will it still fufill the requirements for it to be a Hilbert Space?

Answer 1

A Hilbert space is a set with elements and some operations. In your intended application these elements could be functions. All we care about is that we can add and substract them, we have a $0$ and scalar multiplication (they form a vector space) and we have an inner product $\langle x,y\rangle$ for any $x,y$ in the set, fulfilling some axioms (inspired by those on $\Bbb R^n$, the "mother of all Hilbert spaces"). This inner product allows us to define a norm (a notion of length) by defining $\|x\|=\sqrt{\langle x,x\rangle}$ and a norm allows us to define a distance (metric) on the set by setting $d(x,y)=\|x-y\|$ for this norm from the inner product. And whenever we have a set with a distance we can define a Cauchy sequence. Also when the elements are functions and the inner product is defined by some series or integral, e.g., those are just "implemntation details", as long as the defining axioms are fulfilled (for vector space and inner product) we can talk about Cauchy sequences of those elements. Constant sequences are always Cauchy (regardless of the space), and so always is any convergent sequence (the completeness condition for a Hilbert space says that the reverse must also be true: any Cauchy sequence is convergent).

The elements themselves don't matter, the distance function on them is the crucial element.

Answer 2

I think it is easier to explain a space that wouldn't be Cauchy. We can estimate pi with the below equation. So this equation is a cauchy sequence in the rational numbers space. However pi is not part of the rational numbers space. So rational numbers are not a hilbert space.

Why do we want this?

It makes sense, just like when we scale numbers we want the value also to be in the space. This is just an extra restriction.

You don't want to be doing limit operations within our space and accidentally end up in undefined. If we have this restriction method we can use limits when solving equations.

It is also not saying that every sequance is Cauchy. It is also not saying that sin(X) is cauchy. you can have sin(x) in your function space. That's fine, it doesn't need to approach a limit or anything.