So far I know that in the $2 \times 2$ case, the matrices $A =$
\begin{pmatrix} 0 & a \\ 0 & 0 \\ \end{pmatrix}
\begin{pmatrix} 0 & 0 \\ a & 0 \\ \end{pmatrix}
and
\begin{pmatrix} a & a \\ -a & -a \\ \end{pmatrix}
Satisfy $A^2 = 0$, but I'm not sure if this is all of them. I don't think there should be any others.
Also, are there $2 \times 2$ matrices which satisfy $A^3 = 0$ or $A^4 = 0$ or $A^5=0$, etc... but not $A^2 = 0$? I want to say "no" but I'm having a hard time explaining why.
The trace and the determinant must be zero. So force the trace to be zero by taking $$A=\pmatrix{a&b\\c&-a}.$$ The square of this is zero iff $-a^2+bc=0$ so we can take $c=-a^2/b$ if $a\ne0$. (You've already done the $a=0$ case.)