Classical Gram-Schmidt orthogonalization produces large backward error

114 Views Asked by At

Consider the vectors $u$ and $v$ given by$$u=(1,1)/\sqrt{2}$$ and $$v=(1+\epsilon,1-\epsilon).$$ The Gram-Schmidt orthogonalization will consist of projection $$\hat{v}=v-(u^Tv)u$$ and consequent normalization $$\tilde{v}=\hat{v}/||\hat{v}||_2.$$ Prove the following conjectures:

Conjecture 1: Use $$fl(u^Tv)=\sum_{i=1}^2u_iv_i(1+2\delta_i)$$ to arrive at $$\hat{v}_j=(v_j(1+\delta_j)-\sum_{i=1}^2(u_i(1+2\hat{\delta}_i)v_i(1+\delta_i))u_j(1+2\hat{\delta}_j)$$ and $$\hat{v}_j=(v_j-\sum_{i=1}^2(u_iv_i)u_j)(1+C\bar{\delta})$$ where C is bounded by entries of $u$ and $v$

Conjecture 2: Use $$fl(u^Tv)=\sum_{i=1}^2u_iv_i(1+2\sigma_i)$$ to arrive at $$fl(||v||_2)=||v||_2(1+\tilde{C}\delta)$$ if $\hat{v}=v(1+C\delta)$ where C is bounded by entries of $u$ and $v$

Conjecture 3: Use Conjecture 1 and 2 to show that when $\epsilon$ is small, the Gram-Schmidt orthogonalization results in a large backward error

Any ideas on how to proceed?


Edit : So here's what happens when you start plugging stuff in and foiling it out. The intuition into coming up with the right formulation is still not clicking but it does say to use "empirical" estimates. I'm still not sure why we need the particular values of u and v for the formulation in conjectures 1 and 2.

$$\hat{v}=v-(u^Tv)u=(1+\epsilon,1-\epsilon)-\frac{2}{\sqrt{2}}(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})=(\epsilon,-\epsilon)$$ and $$\tilde{v}=\frac{\hat{v}}{||\hat{v}||}=\frac{(\epsilon,-\epsilon)}{\sqrt{2}\epsilon}=(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$$

but

$$fl(u^Tv)=\sum_{i=1}^2u_iv_i(1+2\delta_i)=u_1v_1(1+2\delta_1)+u_2v_2(1+2\delta_2) \\=\frac{1+\epsilon}{\sqrt{2}}(1+2\delta_1)+\frac{1-\epsilon}{\sqrt{2}}(1+2\delta_2)$$

So I'm confused as to what I'm supposed to do with $\hat{v}$ and $\tilde{v}$ for the floating point arithmetic part. I mean it's obvious as to why we get those roundoff error expressions, which are added for each algebraic operation(I'm still not sure how they get a 2 in front of delta for some of them), but it makes sense overall.