Perturbing the inverse of a singular matrix

Question

Consider the equation $A.\vec{x} = \vec{b}$, where $A$ is a square singular matrix, which has (potentially degenerate) complex eigenvalues, and a single zero eigenvalue. There is no unique solution to this equation as the inverse of $A$ does not exist.

However, since the set of singular matrices is infinitely thin I can invert $\left[i\epsilon I -A\right]^{-1}$ for small $\epsilon$ and the identity matrix $I$.

My question is this: is the solution to the equation $\left[i\epsilon I -A\right].\vec{x} = \vec{b}$ $\epsilon$-close to solution of $A.\vec{x}=\vec{b}$?

Answer 1

The answer is in general negative. The system $A x = b$ may even have no solutions. You may want to check inequalities regarding error propagation in the solution of linear systems that include the notion of "condition number".

Answer 2

Let me make this very concrete:

Consider $$ A = \pmatrix{0 & 1 \\ 0 & 0 }, b = \pmatrix{3 \\ 0}. $$ The $x$ you (probably) want in this case (i.e., the solution to $Ax = b$) is $\pmatrix{0\\3}$, although $x = \pmatrix{c\\3}$ works for any $c$.

Your perturbed matrix is $$ A' = \pmatrix{\epsilon i & 1 \\ 0 & \epsilon i} $$ and the solution to $A'x' = b$ is $x' = \pmatrix{\frac{3}{\epsilon i} \\ 0}$, whose distance from (any of the possible solutions) $x$ is at least $3$.

N.B. I used $A' = A - i\epsilon I$ rather than your suggested $i\epsilon I - A$, because the former converges to $A$ as $\epsilon$ gets small, but the latter converges to $-A$, so that you'd expect $x'$ to converge to something like $-x$ rather than $x$. You can see this by taking $A = \pmatrix{2 & 0 \\ 0 & 0}$ and $b = \pmatrix{1 \\ 0}$, for instance.