If $A+\epsilon $ (where $\epsilon$ is a perturbation on the order of machine precision,) has an eigenvalue $\lambda$, and $A$ is a normal matrix, then the eigendecomposition of $A$ is $A=Q \Lambda Q^{*}$, where $Q$ is unitary.
Under those conditions, I have seen it written in my textbook that $\|(\lambda I-A)^{-1}\|=\|Q(\lambda I-\Lambda )^{-1} Q^{*}\|$. But I cannot figure out why these two things are equal.
Can someone show me why/how these are equivalent?
Thank you.
The $\|\cdot\|$ denotes the $2$-norm.
This clearly isn't an exact equation, but rather a relationship that holds in the limit as $\| \epsilon \|$ goes to 0.
There's a notational confusion here, in that $\lambda$ in the statement of the problem is an eigenvalue of the perturbed matrix, and $\Lambda$ is the diagonal matrix of eigenvalues of the original matrix. I'll change the notation to use $\mu$ as an eigenvalue of the perturbed matrix $A+\epsilon$, and restate the proposition as
Since $\mu$ is an eigenvalue of the perturbed matrix, it is close to an eigenvalue $\lambda_{i}$ of the original matrix $A$, with corresponding eigenvector $v_{i}$. The perturbation theory for eigenvalues tells us that to first order,
$\mu \approx \lambda_{i} + v_{i}^{T}\epsilon v_{i}$
Thus the smallest eigenvalue of $\mu I - A$ is to first order,
$\mu_{smallest}(\mu I - A) \approx v_{i}^{T} \epsilon v_{i} $
Then to first order,
$\| (\mu I - A)^{-1} \|_{2} \approx \frac{1}{| v_{i}^{T} \epsilon v_{i} |} $
On the other side, the smallest eigenvalue of $\mu I - \Lambda$ is $\mu - \lambda_{i} \approx v_{i}^{T} \epsilon v_{i}$. Thus to first order,
$\| (\mu I - \Lambda)^{-1} \|_{2} = \| Q(\mu I - \Lambda)^{-1} Q^{*} \|_{2} \approx \frac{1}{| v_{i}^{T} \epsilon v_{i} |}$
You can easily check this in MATLAB by creating random symmetric matrices $A$ and $\epsilon$ with $\| \epsilon \| \ll \| A \|$.
Note that this fails if for some reason $v_{i}^{T}\epsilon v_{i}=0$. I believe that repeated eigenvalues in $A$ can also cause problems.