I have some issue understanding some steps of this notes, pag 5:
We want to understand the eigenvalues of the perturbed symmetric matrix $(A+\epsilon B)$. The original matrix $A$ has eigenvalues $\lambda_i$ and $x_i$. The perturbation is a symmetric matrix $B$ such that $\|B\|_2=1$. We denote with $\lambda(\epsilon)$ and $x(\epsilon)$ the perturbed eigenvalues and eigenvector. Then, the eigenvalue equation can be written as: $$ (A+\epsilon B)x(\epsilon) = \lambda(\epsilon)x(\epsilon)$$
I am going to re-do the steps, albeit with inserting some questions in it:
First, we perform Taylor expansion on $\lambda(\epsilon)$ and $x(\epsilon)$.
$$ (A+\epsilon B)(x + x'(0)\epsilon + \frac{1}{2}x''(0)\epsilon^2 + \cdots) =\\ (\lambda + \lambda'(0)\epsilon + \frac{1}{2}\lambda''(0)\epsilon^2 + ...) (x + x'(0)\epsilon + \frac{1}{2}x''(0)\epsilon^2 + ...) $$
Now we take only the terms that are coefficient of $\epsilon$: $$ Bx + Ax'(0) = \lambda'(0)x + \lambda x'(0)$$
Now we multiply on the left by $x^T$ (why?) and obtain: $$ x^TBx + x^TAx'(0) = \lambda'(0)x^Tx + \lambda x^Tx'(0) $$
as $Ax'(0)=\lambda x'(0)$ $$ x^TBx + x^TAx'(0) = \lambda'(0)x^Tx + \lambda x^Tx'(0) $$
from here it is easy for me to go to the equation: $$ \frac{x^TBx}{x^Tx} = \lambda'(0)$$
It follows that: $$ \lambda(\epsilon) - \lambda(0) = \frac{x^TBx}{x^Tx}\epsilon + O(\epsilon^2)$$
Since $|\frac{x^TBx}{x^Tx}| \leq \|B\|_2 $ we have that $$| \lambda(\epsilon) - \lambda(0)| \leq |\epsilon| \|B\|_2 + O(\epsilon^2)$$
And using the assumption that $\|B\|_2 \leq 1$ for the perturbation matrix $B$, we have:
$\|\lambda(\epsilon) - \lambda(0)| = O(\epsilon)$
This shows that a perturbation on a symmetric matrix $A$ is $O(\epsilon)$.
I have some questions that prevents me from a deep understanding of these steps:
- I don't understand how Taylor is used in this case. What is the idea for taking the coefficients of $\epsilon$? I don't understand the difference between $x'(0)$ and $x$?
- Where the symmetry of $A$ has been used?
- Is it correct to say that $\lambda(\epsilon) - \lambda(0) = \lambda'(0)$? Otherwise I do'to see how we can get: $$ \lambda(\epsilon) - \lambda(0) = \frac{x^TBx}{x^Tx}\epsilon + O(\epsilon^2)$$
- We used in the Taylor step the fact that: $Ax'(0)=\lambda x'(0)$. Why? We know that $Ax(0) = \lambda x(0)$, we don't know anything about $x'(0)$.