Uniqueness of solution of system of equations

Question

Suppose we have a general system of $n$ equations and $n$ unkonwns $$Ax=b$$ where we say that the system has a unique solution iff $rank(A)=n$. This is equivalent to saying that $A$ has no 0 eigenvalues. If, however, the eigenvalues are arbitrarily close to 0, does this in any way affect the stability/uniqueness of the solution set? Thanks!

Answer 1

The stability of the solution to the system of linear equations $Ax=b$, where $A$ is a rank-$n$ square matrix of dimension $n$, depends on the condition number of $A$. The condition number is defined by the ratio of the maximum singular value of $A$ to its minimum singular value. A lower condition number yields more stable solutions. To give you some intuition, if $A$ is ill-conditioned (i.e. high condition number), a small variation in $b$ induces a big change in $x$.

Note: You can solve the problem by multiplying the equation by $A'$ on both sides and taking isolating $x$: $$x = (A'A)^{-1}A'b.$$ Because $A'A$ is necessarily Hermitian, its eigenvalues are equal to its singular values. Hence, the condition number of $A'A$ is given by the ratio of the maximum eigenvalue to the minimum eigenvalue.