Why are there infinitely many orthonormal vectors?

Question

By Graham Schmidt process we can create infinitely many orthonormal vectors, but my doubt is that why is it not bounded by the dimensionality of the space ? Intuitively (geometrically) how can we explain existence of infinitely many vectors, since there are only few possible directions to obtain normal to a vector.

Answer 1

Graham-Schmidt kicks out a zero vector whenever the current vector lies in the span of the previously considered vectors. When continuing on to the next step, zero vectors must be discarded (otherwise you'd get a "division by zero" error when trying to divide by its length in the next step).

Specifically, if $u_1,u_2,\dots$ are input vectors and $w_1=u_1, w_2, w_3,\dots$ are outputs, then for each $k$ the span of $\{u_1,\dots,u_k\}$ and the span of $\{w_1,\dots,w_k\}$ are equal. Now orthogonal vectors are linearly independent, so if $u_k$ lies in the span of the first $k-1$ vectors, then when you compute $w_k$, it will turn out that $w_k=0$.

So while you can apply this process to a potentially infinite subset of a finite dimensional vector space, only finitely many output vectors will be nonzero...and yes, the number of nonzero vectors is bounded by the dimension of the space you're working in.

Now if you're working in an infinite dimensional space, Graham-Schmidt can be applied to a (countably) infinite set and will produce infinitely many orthogonal vectors (if the input set is linearly independent). For example: apply Graham-Schmidt to $\{1,x,x^2,\dots\}$ in the space of polynomials $\mathbb{R}[x]$ equipped with some inner product like $\langle f,g \rangle = \int_0^1 f(x)g(x)\,dx$.

Answer 2

In $\mathbb{R}^2$ there is only one orthogonal direction to a given vector and two orthonormal vectors (differing only in sign). In $\mathbb{R}^n$ for $n \geq 3$, given a vector $v$ there is an $n-1$-plane orthogonal to it, and any vector lying on that plane is orthogonal to $v$. There is an infinite number of such vectors, and Graham-Schmidt allows you find some orthogonal basis, but there is an infinite number of such basis, all having the same dimension $n$.