Is there a more convenient method for converting a base to be orthogonal than Gram Schmidt?

Question

I find performing the Gram-Schmidt method a little cumbersome and confusing (easy to make mistakes along the way)

Maybe I just need to practice it more, but are there other more convenient methods which I could use? Or alternatively are there any tricks which could make actually performing it easier?

Answer 1

It's an algorithm fairly well suited to computer-based implementation, and generally a pain in the neck to do by hand.

On the other hand, if you write your basis (assuming we're talking about $\Bbb R^n$ here for the moment) as the columns of a matrix, then doing a little column reduction can make your job easier, with fewer square roots. For instance, if we start with $$ \pmatrix{1 & 2 \\ 2 & 2 } $$ then GS involves a $\sqrt{5}$ to normalize the first vector, and that persists in the computations with the second vector. But if we add the second vector to the first, we get $$ \pmatrix{3 & 2 \\ 4 & 2 } $$ and because $3^2 + 4^2 = 5^2$, the normalization of the first column is really easy: it becomes

$$ \pmatrix{\frac{3}{5} & 2 \\ \frac{4}{5} & 2 }. $$

But the fact is that opportunities like these don't present themselves very often, and it's seldom worth looking for them. On the other hand, doing a bit of "column reduction" to make as many entries zero as possible...that's generally a way to reduce the number of places where you can make numerical/algebraic mistakes while carrying out he rest of G-S.

Following up on @Gimusi's answer:

A typical "step" in G-S consists of having a set (possibly empty) of orthonormal vectors $u_1, \ldots, u_k$ and a set $\{v_{k+1}, \ldots, v_p\}$ that you want to move into the $u$-set. You do this a vector at a time. Letting $v$ denote $v_{k+1}$, you typically do this as follows:

1. for $i = 1, \ldots k$, make $v$ orthogonal to $u_i$ by replacing $v$ with $$ v' = v - (v \cdot u_i) u_i. $$ When you do that, you have $(v' \cdot u_i) = (v \cdot u_i) - (v \cdot u_i) (u_i \cdot u_i)$, and because the $u_i$ are all unit vectors, the last factor is $1$, so $v'$ ends up orthogonal to $u_i$.

2. Once you've made $v$ orthogonal to all the $u_i$s, replace $v$ with $$ v' = \frac{1}{\|v\|} v, $$ a process called "normalizing" it, and let $u_{k+1} $ be this new, unit length $v$ (and delete $v_{k+1}$ from the $v$-set.

Repeat until done.

At each stage, the $u$-set consists of orthogonal unit vectors.

In the alternative version, you simply make the $u$s orthogonal to one another, but not necessarily unit vectors. You then follow this process:

1. for $i = 1, \ldots k$, make $v$ orthogonal to $u_i$ by replacing $v$ with $$ v' = v - \frac{v \cdot u_i}{u_i \cdot u_i} u_i. $$ When you do that, you have \begin{align} (v' \cdot u_i) &= (v \cdot u_i) - \frac{v \cdot u_i}{u_i \cdot u_i} (u_i \cdot u_i)\\ &= (v \cdot u_i) - v \cdot u_i\\ & = 0 \end{align} so that $v'$ is now orthogonal to $u_i$.

2. Once you've made $v$ orthogonal to all the $u_i$s, let $u_{k+1} $ be this new, unit length $v$ (and delete $v_{k+1}$ from the $v$-set.

Finally, when the $v$-set is empty and the $u$-set is full, normalize each vector in the $u$-set (i.e., divide it by its own length).

Notice that until this last step, there's never a square root in sight!

Answer 2

To facilitate the calculation by GS you can

• try at first to find at least 2 or 3 orthogonal vectors by inspection (usually this is not difficult)
• for the others perform GS letting the normalization at the end of the process