Intuition for Gram-Schmidt process

Question

Let V be a finite dinner-product space, given a$ v_1,...v_n$ orthogonal bases, and $w_1,...w_n \in W $be subspace of V. Then $v_1=w_1$, $v_2= w_2-proj_w v_2$, $v_3= w_3-proj_w v_2- proj_w v_3$... I know that $proj_w v_i$ is the vector $v_i$ projected onto W subspace. How to intuitively understand Gram-Schmidt process, why as i gets bigger, we keep on subtracting more? I looked up the diagram on Gram-Schmidt, but still fails to get the sense.

Wiki for Gram-schmidt

Answer 1

When you project a vector onto a subspace (any dimension), and subtract the result, you get a vector orthogonal to the subspace. Projection onto an $n$-dimensional subspace, on the other hand, can be accomplished by adding the projections onto the $n$ individual members of an orthogonal basis.

Perhaps try an example. Project $(x,y,z)$ onto the $2$-dimensional subspace spanned by the unit vectors in the directions of the $x$ and $y$ axes. These two are often denoted $\bf{\vec i}$ and $\bf{\vec j}$. The projection in this case is of course $(x,y,0)$. And that's indeed $(x,y,z)-(x,0,0)-(0,y,0)=(x,y,z)-\operatorname{proj}_{\bf{\vec i}}(x,y,z)-\operatorname{proj}_{\bf{\vec j}}(x,y,z)$. After subtracting, and normalizing, you of course get $\bf{\vec k}=(0,0,1)$, to complete the standard basis.

Answer 2

What helped me was to just follow the algebra.

We claim that $\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u$ is orthogonal to $\mathbf u$.

In which case $\mathbf {u}\cdot(\mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf u) = 0$

Try it out.

$\mathbf{u}\cdot \mathbf{v} - \frac {\mathbf{u}\cdot \mathbf{v}}{\mathbf{u}\cdot \mathbf{u}} \mathbf{u}\cdot \mathbf{u} = \mathbf{u}\cdot \mathbf{v} - \mathbf{u}\cdot \mathbf{v} = 0$

Answer 3

The article you linked is quite good and has a very nice animation

Say have two linearly independent vectors $\mathbf{u_1},\mathbf{u_2}$

You first calculate $proj_{\mathbf{u_1}}(\mathbf{u_2})$, the projection of $\mathbf{u_2}$ to the line generated by $\mathbf{u_1}$ which is calculated by $\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$

Now if we subtract it from $\mathbf{u_2}$ we'll get rid of the "common" part of the two vectors (that's where the Venn's diagram comes into play).

You can now easily prove that $\mathbf{v_1} = \mathbf{u_1},\space \mathbf{v_2} = \mathbf{u_2} -\frac{\mathbf{u_1}\cdot\mathbf{u_2}}{||\mathbf{u_1}||}\mathbf{u_1}$ are orthogonal

How would you construct an inductive proof for $n$ vectors?