Condition Number and Sensitivity of Matrix-Vector Multiplication

Question

Consider a non-singular matrix $A \in \mathbb{R}^{n \times n}$ and vectors $x,b \in \mathbb{R}^n$ such that $Ax = b.$ Intuitively the condition number $\kappa(A)$ captures the rate at which $x$ will change given changes in $b$ (i.e. the backward error). My question is, does the condition number capture any information about the forward error? For instance, if I have a large $\kappa(A)$ will changes in $x$ generally lead to large changes in $b$? It would seem that the fact that $\kappa(A) = \kappa(A^{-1})$ would imply that this is true.

Answer 1

Your intuition is correct. Here is how it's proved.

$ b = A x $ and $ b + \delta\!b = A ( x + \delta\!x ) $ so that $ \delta\!b = A \delta\! x $.

Now, for a vector norm $ \| \cdot \| $ and corresponding consistent matrix norm $ \| \cdot \| $ we know that

$ \| \delta\!b \| = \| A \delta\! x \| \leq \| A \| \| \delta\! x \| $ also $ \| x \| = \| A^{-1} b \| \leq \| A^{-1} \| \| b \| $.

Now,

$ \frac{\| \delta\ b \|}{\| b \|} \leq \| A \| \| A^{-1} \| \frac{ \| \delta\! x \|}{\| x \|} $

It can be shown, through standard techniques, that there are choices of $ x $ and $ \delta\! x $ such that this inequality becomes an equality. You do this by carefully considering the largest and smallest singular vectors.