Two problems from Differential Equations; Dynamical Systems, and an Introduction to Chaos (Morris W. Hirsch,Stephen Smale.Robert L. Devaney), examples page 112,113:
If $$A= \begin{pmatrix} 2 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & -1 & 2 \\ \end{pmatrix}$$ … $\lambda =2 , m_{\lambda}=3$ $rank(A-2I)=2 , n_{\lambda}=1 ,k= m_{\lambda}- n_{\lambda}+1=3$ so $$p= \begin{pmatrix} 1& 0 & 1 \\ -1 & 0 & 0 \\ 0 & -1 & 0 \\ \end{pmatrix}$$ by $v_2=(A-2I)v_3, v_1=(A-2I)v_2, v_3 : \begin{cases} (A-2I)^3v_3= 0\\ (A-2I)^2v_3 \not = 0\\ \end{cases} $But $p^{-1}Ap \not =J$ $$p= \begin{pmatrix} 0& 0 & 0 \\ 0& 1 & 0 \\ 0 & 0 & 1 \\ \end{pmatrix} \not = \begin{pmatrix} 2& 1 & 0 \\ 0 & 2 & 1 \\ 0 & 0 & 2 \\ \end{pmatrix}$$ what’s wrong?!
I will assume you understand how they derived $P$. (If not, please respond).
We have:
$$P= \begin{pmatrix}1& 0 & 1 \\-1 & 0 & 0 \\0 & -1 & 0 \end{pmatrix}$$
This give us:
$$P^{-1}= \begin{pmatrix} 0 & -1 & 0 \\0 & 0 & -1 \\1 & 1 & 0 \end{pmatrix}$$
From this we get:
$$P^{-1}AP = \begin{pmatrix} 0 & -1 & 0 \\0 & 0 & -1 \\1 & 1 & 0 \end{pmatrix} \cdot \begin{pmatrix}2 & 0 & -1 \\0 & 2 & 1 \\-1 & -1 & 2 \end{pmatrix} \cdot \begin{pmatrix}1& 0 & 1 \\-1 & 0 & 0 \\0 & -1 & 0 \end{pmatrix} = \begin{pmatrix}2 & 1 & 0 \\ 0 & 2 & 1 \\0 & 0 & 2\end{pmatrix}$$
This validates the authors' solution.
Did you calculate $P^{-1}$ or the product $P^{-1}AP$ incorrectly?