Consider the matrices below $$A=\begin{bmatrix}9&4&5\\-4&0&-3\\-6&-4&-2\end{bmatrix}$$ and $$B=\begin{bmatrix}2&1&0\\0&2&0\\0&0&3\end{bmatrix}$$
These matrices have the same eigenvalues $\{2,2,-3\}$ and the same Jordan Canonical Form so they are similar.
In trying to find $S$ s.t. $B=S^{-1}AS$ I set $$S=\begin{bmatrix}a&b&c\\d&e&f\\g&h&k\end{bmatrix}$$ and then tried to solve the system of 9 equations with 9 unknowns $$B=S^{-1}AS\Leftrightarrow SB=AS$$ but Matlab showed it is rank deficient so it provided only the zero solution.
How can I find such matrix $S$?
Is there a systematic way to do that in the general case when both $A,B$ are $n\times n$ matrices?
Notice that $B$ is precisely the Jordan form of $A$, so it suffices to find a Jordan basis for $A$.
We have:
$$A - 2I =\begin{bmatrix}7&4&5\\-4&-2&-3\\-6&-4&-4\end{bmatrix},\quad (A - 2I)^2 =\begin{bmatrix}3&0&3\\-2&0&-2\\-2&0&-2\end{bmatrix}$$
so $\ker (A - 2I)^2 = \operatorname{span}\{e_1- e_3, e_2\} = \operatorname{span}\left\{\pmatrix{1 \\ 0\\ -1}, \pmatrix{0\\1\\0}\right\}$.
$$A - 3I = \begin{bmatrix}6&4&5\\-4&-1&-3\\-6&-4&-5\end{bmatrix} \implies \ker (A - 3I) = \operatorname{span}\left\{\pmatrix{-3 \\ 2 \\ 2}\right\}$$
Therefore, one Jordan basis is
$$\left\{(A - 2I)e_2, e_2, \pmatrix{-3 \\ 2 \\ 2}\right\} = \operatorname{span}\left\{\pmatrix{4 \\ -2\\ -4}, \pmatrix{0\\1\\0}, \pmatrix{-3 \\ 2 \\ 2}\right\}$$
So the similarity matrix is
$$S = \begin{bmatrix}4&0&-3\\-2&1&2\\-4&0&2\end{bmatrix}$$