Gram-Schmidt Process and Orthogonal Components

Question

Let the Gram-Schmidt process transform the vector system $(a_{1}, ..., a_{n})$ into the system $(b_{1}, ..., b_{n})$. Show that the vector $b_{k}$ is the orthogonal component of the vector $a_{k}$ with respect to the linear span of the system $(a_{1}, ..., a_{k-1})$ ($k > 1$)

---

I tried to first notice that $\mathrm{Span}(a_{1}, ..., a_{k}) = \mathrm{Span}(b_{1}, ..., b_{k})$ ($\forall k$) and the collection $(a_{i})_{i=1}^{k}$ will be pairwise orthogonal (with norm $1$) which follows from the Gram-Schmidt process. However, I'm not sure how to proceed. As well, I'm not sure what the "orthogonal component" of the vector $a_{k}$ refers to. I was thinking of the orthogonal complement to the vector $a_{k}$, but that doesn't make much sense.

Thanks for the help.

Answer 1

By definition of the Gram-Schmidt process without normalisation, $b_k$ is obtained from $a_k$ by subtracting its projection on the linear span of $b_1,\ldots,b_{k-1}$, or of $a_1,\ldots,a_{k-1}$ (by construction, these spans are the same). Then $b_k$ is also the projection of $a_k$ on the orthogonal complement of that span (since any vector is the sum of its projections on two orthogonally complementary subspaces). So this is just the definition of the Gram-Schmidt process.

Note that after normalisation, $b_k$ will no longer be just the projection of $a_k$ on the orthogonal complement, though it will still be in that orthogonal complement.

Answer 2

I think the key point you're missing, as you've noted, is figuring out what the "orthogonal component" means. The easiest way to see this is to consider $\mathbb{R}^3$. Let $(a_1, a_2, a_3)$ be any three vectors in $\mathbb{R}^3$ such that $a_1, a_2$ are in the $xy$-plane.

Then consider doing Gram-Schmidt, and stopping at the moment you determine $b_2$. Now we have an orthonormal basis $(b_1, b_2)$ for $\mathbb{R}^2 = \text{span}(b_1, b_2)$ (namely the $xy$-plane). In general, $a_3$ will not be orthogonal to the $xy$-plane; it just won't be coplanar (since then the original 3 vectors is not a basis). The "orthogonal component" of $a_3$ with respect to $\text{span}(b_1,b_2)$ is essentially the z-component of $a_3$, i.e. the perpendicular of $a_3$.

Equivalently, the orthogonal component is what's left of $a_k$ after subtracting off the projection of $a_k$ onto the span of $b_1, \dots, b_{k-1})$.

Answer 3

If you have a vector $v$ in a vector space $V$ and a vector subspace $W \subset V$ (in this case the one spanned by the vectors $a_1,\dots,a_{k-1}$) you can decompose your vector $v$ into $v_1$ and $v_2$ (meaning $v = v_1 + v_2$) such that $v_1 \in W$ and $\langle v_2, w\rangle = 0$ for every $w \in W$. $v_2$ is called the orthogonal component.

Now that the definition is clear it should be easy to understand what is going on, indeed what you are really doing when you apply the G-S process is throwing away the components that are parallel to the subspace generated by the vectors listed before and keep the orthogonal components.