How the eigenvectors change if you change the matrix

Question

Let $A$ be an n x n matrix with n distinct eigenpairs $(\lambda_1,v_1),(\lambda_2,v_2),...(\lambda_n,v_n)$. Suppose we have $|\lambda_1| \gt |\lambda_2| \gt ... \gt |\lambda_n|$.

Let $B = A - \frac{\lambda_1 v_1 v_1^T}{v_1^Tv_1}$. Show that $B$ has the same eigenvalues and eigenvectors as $A$, except $\lambda_1$ is replaced by zero.

I can easily to show that $Bv_1 = 0$. But I'm having trouble showing the statement for $v_i$ $(i \neq 1)$. I already know that $Bv_i = (A-\frac{\lambda_1 v_1 v_1^T}{v_1^Tv_1})v_i = Av_i - \frac{\lambda_1 v_1 v_1^T}{v_1^Tv_1}v_i = \lambda_iv_i - \frac{\lambda_1 v_1 v_1^T}{v_1^Tv_1}v_i$.

So it remains to show that $\frac{\lambda_1 v_1 v_1^T}{v_1^Tv_1}v_i$ = $\lambda_iv_i$. I tried to show it by brute force but just ended up being confused. I appreciate any help!

Answer 1

In this case, $A$ has to be symmetric. Then all eigenvectors are pairwise orthogonal. Then $v_1^Tv_i = 0$ for $i \neq 1$. The result follows immediately.

Answer 2

In general, this does only work for symmetric matrices $A$. This is because the eigenvectors of symmetric matrices form an orthogonal basis, i.e.

$$v_i^\top v_1=0\quad\text{for all $i\not=1$}.$$

To see that this does not work for non-symmetric matrices in general, see the answer of @Siong Thye Goh.

Answer 3

Let $A= \begin{bmatrix}2 & 1 \\ 0 & 1 \end{bmatrix}$,

then we have $$A\begin{bmatrix} 1 \\ 0\end{bmatrix}= 2\begin{bmatrix} 1 \\ 0\end{bmatrix}$$

and $$A\begin{bmatrix} 1 \\ -1\end{bmatrix} = 1 \begin{bmatrix} 1 \\ -1\end{bmatrix} $$

$$B=\begin{bmatrix} 0 & 1 \\ 0 & 1\end{bmatrix}$$

then the eigenvalues are $0$ and $1$,

we have $$B\begin{bmatrix} 1 \\ 0\end{bmatrix}= 0\begin{bmatrix} 1 \\ 0\end{bmatrix}$$

but $$B\begin{bmatrix} 1 \\ 1\end{bmatrix} = 1 \begin{bmatrix} 1 \\ 1\end{bmatrix} $$

Without the symmetric condition, this is not true. If it is symmetric, then use the property that eigenvector corresponding to distinct eigenvalue of a symmetric matrix is orthogonal.