Rounding error of matrix multiplication when one of the matrices is orthogonal

407 Views Asked by At

I am studying Scientific computing from Biswa Nath Datta's Numerical Linear Algebra and Applications and there is a corollary after explaining matrix multiplication rounding error described below.
if $A \in R^{n*n}$ and $Q \in R^{n*n}$ is orthogonal then
$||fl(QA)-QA||_f \leq n \mu||A||_f$
the older version of book mentioned the proof is in wilkinson 1965 but i didn't find anyhting related there.I am struggling on this for some weeks and i don't undestand it or can success in proving it i was wondering if anyone knows any relatable hints. thanks.

1

There are 1 best solutions below

2
On BEST ANSWER

In the main contributions, the multiplication error $$ |fl(q·a)-q·a|\le \mu|q|·|a| $$ generalizes to the error of the scalar product as $$ |fl(\langle q,a\rangle) - ⟨q,a⟩|\le\mu⟨|q|,|a|⟩ $$ ignoring the errors that occur during addition, or integrating them into the error of FMA (fused multiply-add) operations. By Cauchy-Schwarz, $$ ⟨|q|,|a|⟩\le\|q\|_2\|a\|_2 $$ If $q$ is the row or column of an orthogonal matrix, then its norm is one.

The Frobenius norm is the square root over the square sum over all matrix entries. The matrix entries of the error difference $fl(QA)-QA$ have all the form $\delta ⟨|q|,|a|⟩$, with $|\delta|\le \mu$. Each column $a_{:,k}$ of $A$ occurs $n$ times in the product entries, so $$ \|fl(QA)-QA\|_F^2 \le\sum_{i,k} \delta_{i,k}^2⟨|q_{i,:}|,|a_{:,k}|⟩^2 \le n\mu^2\sum_k\|a_{:,k}\|_2^2=n\mu^2\|A\|_F^2 $$ This gives the even smaller bound $$ \|fl(QA)-QA\|_F\le\sqrt{n}\mu\|A\|_F $$