In A Generalized Representer Theorem page 6, the author wriites:
...Given $x_1,...,x_m$, any $f\in\mathcal{F}$ can be decomposed into a part that lives in the span of the φ($x_i$) and a part which is orthogonal to it: $$f=\sum_{i=1}^m \alpha_iφ(x_i) + v $$...
And this is confusing to me. The entire theorem is to show that $f$ is in fact finite dimensional but the proof seems to assume it at the beginning. My question is, given a infinite-dimensional function class $\mathcal{F}$, why can we decompose its element $f$ into finite partial sums? Thanks in advance.
This follows from a basic (but none trivial fact) about Hilbert Spaces: Let $\mathcal{S} \subset \mathcal{F}$ be a finite dimensional subspace of a Hilbert space $\mathcal{F}$. Then $\mathcal{S}$ is closed (since it's finite dimensional). Thus since $\mathcal{F}$ is a Hilbert space (not just any inner product space), we have the decomposition: $$ \mathcal{F} = \mathcal{S} \oplus \mathcal{S}^{\perp} $$ Where $\mathcal{S}^{\perp} = \{x \vert \; \forall y \in \mathcal{S} \; \langle x,y \rangle = 0 \}$. This Gives the decomposition you mentioned (i.e. decomposing a vector in $\mathcal{F}$ into a finite dimensional part in $\mathcal{S}$ and one in it's orthogonal complement).