We are given a matrix $A = \begin{bmatrix} -1 & -3 & -9\\ 0 & 5 & 18\\ 0 & -2 & -7 \end{bmatrix}$ and we have to find a transition matrix for it.
I have computed a characteristic polynomial $(\lambda + 1)^3$, so the eigevnalue with algebraic multtiplicity 3 is $\lambda = -1$
Then, computing $A - \lambda \mathbb{I} = \begin{bmatrix} 0 & -3 & -9\\ 0 & 6 & 18 \\ 0 & -2 & -6 \end{bmatrix}$ and $(A - \lambda \mathbb{I})^2 = 0$
For constructing a transition matrix we have to find $ v \in Ker(A - \lambda \mathbb{I})^2$, $ v \notin Ker(A - \lambda \mathbb{I})$. This is true for example for $v = (0,0,1)$, and then computing $(A - \lambda \mathbb{I}) v = (-9, 18, -6)$, which are the columns for the transition matrix.
I don't know how to find the third column for it though, since $(A - \lambda \mathbb{I})^2 v = (0,0,0)$, which is not the right solution. Any help would be appreciated
When finding a transition matrix you first find out for what exponent $(A-\lambda I)^k=0$ which is $k=2$ here. And $k=2$ is the lowest possible exponent because $(A-\lambda I) \ne 0$.
First you now want to find a complement $U$ of $\ker(A-\lambda I)$ in $\ker((A-\lambda I)^2) = \mathbb R^3$ which you did (it has the basis $v=(0,0,1)$, i.e. $U= \text{span}(v)$).
Then we go "one level down" and find a complement of $(A-\lambda I)(U)$ in $\ker(A-\lambda I)$, that is: a vector $w$ in $\ker(A-\lambda I)$ that is not element of $(A-\lambda I)(U) = \text{span}(-9, 18, -6)$. What that means in this case is: we want to find a second eigenvector of $A$ for the eigenvalue $\lambda$. We see that $w=(0,3,-1)$ satisfies $(A-\lambda I)w=0$ but $w$ does not lie in $\text{span}(-9, 18, -6)$.
We now have three vectors consisting of a basis that can be used as the columns of a transition matrix: Set $$S= \begin{pmatrix}-9&0&0\\18&0&3\\-6&1&-1 \end{pmatrix}$$
Then $$S^{-1}AS=J=\begin{pmatrix}-1&1&0\\0&-1&0\\0&0&-1 \end{pmatrix}$$ which is the Jordan normal form of $A$.
In terms of the change of basis the matrix $S$ represents the identity operator with respect to the bases $B= ((-9, 18, -6), (0,0,1), (0,3,-1))$ and $C=((1,0,0), (0,1,0), (0,0,1))$. The base $B$ was chosen so that the matrix $J$ has the given form: we put the two "corresponding" vectors $(A-\lambda I)(0,0,1)$ and $(0,0,1)$ in the first two columns to form the block of size two and the last eigenvector $(0,3,-1)$ in the third column to form the block of size one.