Gram-Schmidt Process: How does it fail?

Question

I am confused about the Gram-Schmidt process for a linearly dependent set. I can't seem to grasp why it would cause the linearly dependent vectors to equal zero. Can someone please explain?

Answer 1

Try applying it to the vectors $\pmatrix{1\\0\\0}, \pmatrix{1\\1\\0}, \pmatrix{2\\1\\ 0}$. It should give you the "aha" you need.

Even simpler, try $\pmatrix{1\\1}, \pmatrix{2\\2}$.

---

Suppose you've been through $i-1$ steps of GS, and your vectors $v_1, \ldots, v_{i-1}$ have been converted into vectors $$ u_1, \ldots, u_{i-1} $$ which are all unit vectors (so $u_j \cdot u_j = 1$) and pairwise orthogonal, so $u_j \cdot u_k = 0$ for $j \ne k$, and whose span is the same as that of $v_1, \ldots, v_{i-1}$.

What happens, in GS, to the vector $v_i$, if it's in the span of $v_1, \ldots, v_{i-1}$ (i.e., if the vectors $v_1, \ldots, v_i$ are dependent, while $v_1, \ldots, v_{i-1}$ are independent?

Well, being in the span of $v_1, \ldots, v_{i-1}$ means that you're in the span of $u_1, \ldots, u_{i-1}$. So $$ v_i = c_1 u_1 + \ldots + c_{i-1}u_{i-1}. $$ What happens to $v_i$ when you apply the first step of GS? You convert $v_i$ to $$v_i - (v_1\cdot u_1)u_1 + \ldots + v_1 \cdot u_{i-1}i_{i-1}),$$ right? Well, what is $v_1 \cdot u_1$?

\begin{align} v_1 \cdot u_1 & = (c_1 u_1 + \ldots + c_{i-1}u_{i-1}) \cdot u_1 & \text{Why?} \\ & = c_1 u_1 \cdot u_1 + \ldots + c_{i-1}u_{i-1} \cdot u_1 \text{why?} \\& = 1c_1 + \ldots + 0c_{i-1} \text{why?} \\ & = c_1 \end{align}

So What do you get after that first step of GS?

Answer 2

In the middle of Gram-Schmidt, you have your current list of (orthogonal) vectors $\{e_i\}$, and a new vector $v$. Then you subtract off the part of $v$ which can be constructed from the $\{e_i\}$, and if $v$ is not in the span of $\{e_i\}$, you are left with the part which is orthogonal to all of the $\{e_i\}$ vectors.

But if $v$ is in the span of the $e_i$, then the "part of $v$ which can be constructed from the $\{e_i\}$" is in fact just $v$ itself, so you get a 0 vector when you subtract that off.