I am reading the technical report by Leo Breimann entitled "Some Infinity Theory for Predictor Ensembles".
On page 4 he has the following:
Definition 1 Let $L_2(P)$ be the space of functions on $\mathbb{R}^D$ that are square - integrable wrt $P(dx)$. A set of functions $F$ in $L_2(P)$ will be called complete if the $L_2$-closure of all finite linear combinations of functions in $F$ by $L_2(F)$, equals $L_2(P)$.
And then the following property:
Property 1 A set of functions, $F$, is complete iff there is no non-zero fucntion $g$ in $L_2(P)$ s.t. $\langle g,f \rangle = 0 \forall f\in F$, where $\langle g,f \rangle = \int f(x)g(x)P(dx).$
I do not understand how Property 1 works. As far as my intuition goes, a zero inner product implies orthogonality. But any vector/function space should be able to be decomposed into an orthogonal set of basis vectors.
Thus property 1 as I understand seems to suggest that there cannot exist a function $g$ which makes the inner product 0,and so by implication there cannot exist an orthogonal set of basis vectors on a complete function space?
So property 1 seems to link completeness and orthogonality in a way which I can't fully grasp. I was hoping someone here could help clarify this understanding for me.
Link to paper: https://statistics.berkeley.edu/tech-reports/579
As the comment has pointed out
Thus there is allowed to exist a "zeroing" function for some $f$ but it cannot be true "for all" $f$, otherwise there will be "holes" in the space naturally.