Matrix with all eigenvalues $0$ but not triangular?

Question

Is the situation described in the title achievable? I am looking for a $3\times 3$ case specifically.

Answer 1

To construct examples of this, it's useful to take the $2\times 2$ case and extend it to larger sizes

First observe that there are no diagonalizable matrices with this property.

Thus, such matrices, if they exist are similar to either

$\begin{bmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 0 & 0 & 0\end{bmatrix}$ or $\begin{bmatrix}0 & 1 & 0\\ 0 & 0 & 0\\ 0 & 0 & 0 \end{bmatrix}$.

The matrix $\begin{bmatrix} 2 & 0 & -1\\ -4 & -1 & 1\\ 4 & 1 & -1\end{bmatrix}$ is an example of a matrix similar to the first of these. To find it I merely multiplied on the left by an (almost) random matrix and on the right by the inverse of this matrix.

Getting back to the $2\times 2$ case, it's useful to look at matrices similar to $\begin{bmatrix} 0 & 1\\ 0 & 0\end{bmatrix}$. Call this matrix $J_1$.

Take an arbitrary invertible two by two matrix $\begin{bmatrix} a & b\\c & d\end{bmatrix}$, say $P_1$, (its inverse is well known) and find the product $P_1J_1\left(P_1\right)^{-1}$. Now just force this product to not be triangular (it's possible). Finally, to extend it to the $n\times n$ (with $n\ge 3$) case just consider $$J_2=\begin{bmatrix} 0 & 1 & 0 &\ldots & 0\\ 0 & 0 & 0 & \ldots & 0\\\vdots & \vdots &\ddots & \ldots & \vdots\\ 0 & 0 & 0 & \ldots & 0\end{bmatrix}$$ and for $P_2$ take $$\begin{bmatrix} a & b & 0 & 0 &\ldots & 0\\ c & d & 0 & 0 &\ldots & 0\\ 0 & 0 & 1 & 0 &\ldots & 0\\ 0 & 0 & 0 & 1 & \ldots & 0\\\vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 &\ldots & 1\end{bmatrix}_.$$

The matrix $P_2J_2\left(P_2\right)^{-1}$ is a matrix with the desired property.

Answer 2

The triangular matrices are not normal in general. Only the diagonal matrices are. So take a conjugate of $$A=\begin{bmatrix} 0 & 1 & 1\\0 & 0 &1\\0 & 0 &0\end{bmatrix}$$ For example, consider $PAP^{-1}$ with $$P=\begin{bmatrix} 1 & 0 & 0\\0 & 1 &1\\1 & 0 &1\end{bmatrix}$$ which happens to be $$PAP^{-1}=\begin{bmatrix} 0 & 1 & 0\\-1 & 0 &1\\0 & 1 &0\end{bmatrix}$$