The Fine-Herstein theorem says that the number of nilpotent $n\times n$ matrices over $\mathbb{F}_q$ is $q^{n^2-n}$. I am trying to verify this for the case $n=3$ using the orbit-stabilizer theorem. Here is my method:
- Knowing that the minimal polynomial is $x^m$ for $m\leq n$, I determine the possible rational canonical forms:
$$\left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right), \qquad \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right), \qquad \left( \begin{array}{ccc} 0 & 0 & 0 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right)$$
- Let $\mbox{GL}_n(\mathbb{F}_q)$ act by conjugation on each of these matrices, then the sum of the sizes of the three orbits is the number of nilpotent matrices.
We know that $\left| \mbox{GL}_n (\mathbb{F}_q) \right| =(q^n-1)\cdots (q^n-q^{n-1})$, so that all the work goes in to finding the size of the stabilizer.
This method worked like a charm for the $n=2$ case, but I am having trouble finding what an element of the stabilizer looks like in each case for $n=3$. Is this possible to do just by conjugating by a general matrix and setting it equal to the class representative?
Hint. Try to find the form of the invertible matrices which commute with each of the matrices you listed. There are $(q^3-1)(q^3-q)(q^3-q^2)$, $q^3(q-1)^2$, respectively $q^2(q-1)$ such matrices in each case. Now you get $$1+\dfrac{(q^3-1)(q^3-q)(q^3-q^2)}{q^3(q-1)^2}+\dfrac{(q^3-1)(q^3-q)(q^3-q^2)}{q^2(q-1)}=q^6$$ nilpotent matrices.