I'm currently baffled as I thought that the order of eigenvectors within the basis of a JNF decomposition doesn't matter. I may have a made a mistake in my working, but if not, is there a general rule for the order? This question specifically concerns a case where there is only one eigenvalue and therefore doesn't seem to be a clear order to put the eigenvectors in.
We have:
$$ \begin{equation*} A = \begin{bmatrix} 3 & 0 & 0 \newline 0 & 4 & -1 \newline 0 & 1 & 2 \newline \end{bmatrix} \end{equation*} $$
giving us the following characteristic polynomial:
$$ det(A - \lambda I) = (-\lambda + 3)(\lambda^{2} - 6\lambda + 9) $$
$$ \therefore \lambda_{1} = 3 \ \textrm{(with algebraic multiplicity 3)}$$
Calculating the eigenvectors, we get
$$ \begin{equation*} E_{\lambda_{1}} = span \{ \begin{bmatrix} 1 & 0 & 0 \newline \end{bmatrix}^{T}, \begin{bmatrix} 0 & 1 & 1 \end{bmatrix}^{T} \} \end{equation*} $$
then, using
$$ \begin{equation} \tag{∗} (A - \lambda I)v_{n+1} = b_{n} \end{equation} $$
we can generate our third eigenvector:
$$ \left[ \begin{array}{ccc|c} 0 & 0 & 0 & 1 \newline 0 & 1 & -1 & 0 \newline 0 & 1 & -1 & 0 \end{array} \right] \xrightarrow{\text{RREF}} \left[ \begin{array}{ccc|c} 0 & 1 & -1 & 0 \newline 0 & 0 & 0 & 1 \newline 0 & 0 & 0 & 0 \end{array} \right] $$
We now have three eigenvectors with which we can form our basis, $B$:
$$ b_{1} = \begin{bmatrix} 0 \newline 1 \newline 1 \end{bmatrix}, \ b_{2} = \begin{bmatrix} 1 \newline 0 \newline 0 \end{bmatrix}, \ b_{3} = \begin{bmatrix} 0 \newline 1 \newline 0 \end{bmatrix} $$
I originally assumed that the order that the vectors are generated in by (∗) would form a JNF:
$$ \begin{align} J &= B^{-1}AB \newline &= \begin{bmatrix} 0 & 1 & 0 \newline 1 & 0 & 1 \newline 1 & 0 & 0 \newline \end{bmatrix}^{-1} \begin{bmatrix} 3 & 0 & 0 \newline 0 & 4 & -1 \newline 0 & 1 & 2 \newline \end{bmatrix} \begin{bmatrix} 0 & 1 & 0 \newline 1 & 0 & 1 \newline 1 & 0 & 0 \newline \end{bmatrix} \newline &= \begin{bmatrix} 3 & 0 & 1 \newline 0 & 3 & 0 \newline 0 & 0 & 3 \newline \end{bmatrix} \end{align} $$
but this is clearly not in JNF.
However, with
$$ B = [b_{1}, b_{3}, b_{2}] = \begin{bmatrix} 0 & 0 & 1 \newline 1 & 1 & 0 \newline 1 & 0 & 0 \newline \end{bmatrix} $$
we have
$$ \begin{align} J &= B^{-1}AB \newline &= \begin{bmatrix} 0 & 0 & 1 \newline 1 & 1 & 0 \newline 1 & 0 & 0 \newline \end{bmatrix}^{-1} \begin{bmatrix} 3 & 0 & 0 \newline 0 & 4 & -1 \newline 0 & 1 & 2 \newline \end{bmatrix} \begin{bmatrix} 0 & 0 & 1 \newline 1 & 1 & 0 \newline 1 & 0 & 0 \end{bmatrix} \newline &= \begin{bmatrix} 3 & 1 & 0 \newline 0 & 3 & 0 \newline 0 & 0 & 3 \end{bmatrix} \end{align} $$
This doesn't make much sense to me - is there a general rule for the order that the basis must be in?
The basis can be rearranged in any way that rearranges only the order of the Jordan blocks, but does not rearrange the basis elements within the blocks themselves. That is, basis elements coming from the same Jordan chain must stay in the same order, but the order of different Jordan chains can be changed.
Both $b_1$ and $b_2$ are true eigenvectors of your matrix, but $b_2$ contains some extra information about $A$ since $(A-3I)b_3 = b_2$, so $b_2$ and $b_3$ must be kept in order with a Jordan block describing this structure, so the only choice of rearranging the basis is whether the trivial Jordan block corresponding to $b_1$ comes before the $b_2,b_3$ block, or whether the $b_2,b_3$ block comes before the trivial $b_1$ block. Explicitly, the two valid Jordan forms are $$ J_1 = \begin{pmatrix}3&0&0\\0&3&1\\0&0&3\end{pmatrix}, \quad B_1 = (b_1|b_2|b_3) \quad\text{and}\\ J_2 = \begin{pmatrix}3&1&0\\0&3&0\\0&0&3\end{pmatrix}, \quad B_2 = (b_2|b_3|b_1). $$ The ordering of $b_2$ and $b_3$ is fixed since they come from the same Jordan chain.