Trouble with the Gram-Schmidt Orthogonalization Derivation

41 Views Asked by At

In the book "Linear Algebra as an Introduction to Abstract Mathematics", Pages 102-103 outline the constructive proof used to illustrate the feasibility of generating an orthonormal list of vectors when supplied with a list of linearly independent vectors (i.e. the Gram-Schmidt orthogonalization procedure).

More formally:

If $(v_1,...,v_m)$ is a list of linearly independent vectors in an inner product space $V$, then there exists an orthonormal list $(e_1,...,e_m)$ such that for all $k=1, ..., m$:

$$\text{span}(v_1,...,v_k) = \text{span}(e_1,...,e_k)$$

The author provides three vectors to the reader (the third of which is a generalization of the second vector provided).

  1. $e_1 = \frac{v_1}{\lVert v_1 \rVert}$

  2. $e_2 = \frac{v_2-\langle v_2,e_1 \rangle e_1}{\lVert v_2 - \langle v_2,e_1 \rangle e_1 \rVert}$

  3. $e_k = \frac{v_k - \langle v_k, e_1 \rangle e_1 - \langle v_k,e_2 \rangle e_2 - ... - \langle v_k, e_{k-1}\rangle e_{k-1}}{\lVert v_k - \langle v_k, e_1 \rangle e_1 - \langle v_k,e_2 \rangle e_2 - ... - \langle v_k, e_{k-1}\rangle e_{k-1} \rVert}$

Side note: the second and third equations are constructed using orthogonal decomposition.

For the purpose of this question, I am going to use the author's generalization formula for $e_k$ to produce what $e_3$, in the author's style, would look like.

$$e_3 = \frac{v_3 - \langle v_3, e_1 \rangle e_1 - \langle v_3,e_2 \rangle e_2}{\lVert v_3 - \langle v_3, e_1 \rangle e_1 - \langle v_3,e_2 \rangle e_2 \rVert}$$


I wanted to derive this setup on my own and then compare my results to the author's...and this is where the confusion arises.

My $e_1$...same as the author's:

$$e_1 = \frac{v_1}{\lVert v_1 \rVert}$$

My $e_2$...slight difference in symbols but equivalent because $\lVert e_1 \rVert^2 = 1$

$$e_2 = \frac{v_2-\langle v_2,e_1 \rangle \frac{e_1}{\lVert e_1 \rVert^2}}{\Bigl\lVert v_2 - \langle v_2,e_1 \rangle \frac{e_1}{\lVert e_1 \rVert^2} \Bigl \rVert}$$

Now here is where the issue arises, and I cannot resolve the differences between my $e_3$ and the author's:

$$ e_3=\frac{v_3 -\big(\langle v_3, e_1 \rangle + \langle v_3, e_2 \rangle\big) \frac{e_1 +e_2}{\lVert e_1 + e_2 \rVert^2}}{\Bigl\lVert v_3 -\big(\langle v_3, e_1 \rangle + \langle v_3, e_2 \rangle\big) \frac{e_1 +e_2}{\lVert e_1 + e_2 \rVert^2}\Bigl\rVert}$$

So, specifically, I cannot determine if the following values are equivalent:

$$\color{red}{\text{Mine}}\ \ v_3 -\big(\langle v_3, e_1 \rangle + \langle v_3, e_2 \rangle\big) \frac{e_1 +e_2}{\lVert e_1 + e_2 \rVert^2} \overset{?}{=} v_3 - \langle v_3, e_1 \rangle e_1 - \langle v_3,e_2 \rangle e_2 \ \ \color{blue}{\text{ Author's}}$$


I'll quickly walk through how I arrived at my version of $e_3$.

Firstly, I employed orthogonal decomposition on $v_3$ and the vector $e_1 + e_2$ in order to generate a perpendicular vector (which for the moment we'll call $d$).

$$v_3 = \alpha (e_1 +e_2) + d$$ where $\alpha \in \mathbb R$. That is to say, $v_3$ can be composed into a scalar multiple of $e_1+e_2$ and a vector $d$ such that $d\ \bot\ (e_1 + e_2)$.

Solving for $\alpha$ by setting up the following equation (which uses the inner product definition of orthogonality): $$ 0 = \langle v_3 - \alpha (e_1 + e_2), e_1+e_2 \rangle$$

I arrived at $$\alpha=\frac{\langle v_3, e_1 \rangle + \langle v_3,e_2\rangle}{\langle e_1, e_1 \rangle+\langle e_1, e_2 \rangle+\langle e_2, e_1 \rangle+\langle e_2, e_2\rangle}$$

Because $e_1 \ \bot \ e_2$, this simplifies to:

$$\alpha=\frac{\langle v_3, e_1 \rangle + \langle v_3,e_2\rangle}{\langle e_1, e_1 \rangle+\langle e_2, e_2\rangle}$$

Now, once again, because $e_1 \ \bot \ e_2$, we can employ the Pythagorean Theorem by recognizing that $\langle e_1, e_1 \rangle = \lVert e_1 \rVert^2$ and $\langle e_2, e_2 \rangle = \lVert e_2 \rVert^2$. Therefore...

$$\alpha=\frac{\langle v_3, e_1 \rangle + \langle v_3,e_2\rangle}{\lVert e_1 + e_2 \rVert^2}$$

Returning to $v_3 = \alpha (e_1 +e_2) + d$ and solving for $d$:

$$d = v_3 - \frac{\langle v_3, e_1 \rangle + \langle v_3,e_2\rangle}{\lVert e_1 + e_2 \rVert^2} (e_1 + e_2)$$

Any help would be greatly appreciated as I have not been able to figure it out myself. Cheers~