Classical and modified Gram–Schmidt

Question

If apply classical Gram–Schmidt on a 2 by 2 matrix, and the $q_1$ and $q_2$ I found are not orthogonal. Does that mean I need to apply modified Gram–Schmidt on it? Or does modified Gram–Schmidt can only be applied on matrices which are bigger than 3 by 3?

Answer 1

The classical Gram-Schmidt (CGS) and modified Gram-Schmidt (MGS) processes lead to the same result in exact precision arithmetic. In finite-precision arithmetic, MGS is more numerically stable than the classic process.

For a given square matrix $A$ of size $n$, with linearly independent columns $a_1, ..., a_n$, the algorithms can be written is pseudocode as:

CGS vs. MGS

Note that for $n=2$, the methods coincide even in finite-precision arithmetic. The error in enforcing orthogonality of vectors $q_i$ found by MGS depends not only on machine precision but also on the condition number $\kappa(A)$ of the matrix.

For ill-conditioned matrices other orthogonalization methods are recommended, such as the Householder of the Givens algorithm. These methods produce vectors that are orthogonal up to machine precision. For further information on error bounds of orthogonalization schemes, see The influence of orthogonality on the Arnoldi method by Braconnier, Langlois and Rioual (Linear Algebra and its Applications, 2000).