Various Interpretations of Condition Number

Question

Condition number of a matrix $A$ signifies how quickly solution $x$ changes in $Ax = b$ as we make small changes in $b$. This is given by $||A||.||A^{-1}||$. When we consider spectral norm, this turns out to be the ratio of largest singular value of $A$ to the smallest singular value of $A$. This ratio of singular values can be interpreted as how deformed an ellipsoid we get when we apply $A$ to the sphere. I don't exactly understand how this interpretation of condition number is connected to the earlier interpretation where we are concerned about the change in solution $x$. I understand the connection algebraically and I am looking for a more intuitive explanation.

Answer 1

Once you've convinced yourself that the condition number is given by $\|A\| \cdot \|A^{-1}\|$ (using a derivation such as that presented here), it suffices to note that $$ \|A\| = \max_{\|x\| = 1} \|Ax\| = \sigma_1(A)\\ \|A^{-1}\| = \left[\min_{\|x\| = 1}\|Ax\|\right]^{-1} = \frac 1{\sigma_n(A)} $$ Note that $\{x : \|x\| = 1\}$ is the unit sphere, and $\{Ax: \|x\| = 1\}$ is the ellipsoid we get by applying $A$. By our above definition, $\|A\| = \max_{\|x\| = 1} \|Ax\|$ gives us the greatest distance of a point on the ellipsoid from the origin, i.e. the length of the "major radius" of the ellipsoid. $\min_{\|x\| = 1} \|Ax\|$ gives us the least distance of a point on the ellipsoid from the origin, i.e. the length of the "minor radius" of the ellipsoid. The ratio of these two length is a measure of how deformed the ellipsoid is.