Find endomorphisms $α$ and $β$ of $V$ satisfying the condition that $αβ$ is not nilpotent but $cα+dβ$ is nilpotent for all $c, d ∈ \mathbb{R}$

Question

Let $V = \mathbb{R}^3$. Find endomorphisms $α$ and $F$ of $V$ satisfying the condition that $αβ$ is not nilpotent but $cα+dβ$ is nilpotent for all $c, d ∈ \mathbb{R}$.

I tried $(cα+dβ)^2=c^2α^2+cd(αβ+βα)+d^2β^2=0$. But that don't seem to work, and I don't have another idea that guess and try.

Thanks.

Answer 1

I will denote by $A$, $B$ the desired endomorphisms. The condition $aA+bB$ nilpotent for all $a,b\in \mathbb R$ implies $$ 0=(aA+bB)^3 = (aA+bB)(aA+bB)(aA+bB) \\= a^3A^3 + a^2b( A^2B + ABA + BA^2) + ab^2(B^2A+BAB+A^2B) + b^3 B^3, $$ hence $$ A^3 =0, \\ A^2B + ABA + BA^2=0, \\ B^2A+BAB+A^2B=0,\\ B^3=0. $$ The first and the latter condition imply that $A,B$ have to be nilpotent.

Moreover, the condition $AB$ not nilpotent implies $A^2\ne0$ and $B^2\ne0$, otherwise the second or third line would yield $ABAB=0$.

Following Travis hint, you have to choose $$ A=\pmatrix{ 0 & 1 & 0\\ 0 & 0 & 1 \\ 0 & 0 & 0 }, $$ and $B$ similar to $A$.

---

After fiddling around with the conditions on $B$ implied by $A^2B + ABA + BA^2=0$, I found one matrix $B$ that apparently does fulfill all the conditions $$ B = \pmatrix{0&1&0\\ 1&0&1\\ 0&-1&0\\}. $$ Still some high-level inside into this problem is highly appreciated :)

Answer 2

Hints

1. Since $c \alpha + d \beta$ is nilpotent for all $c, d$, in particular $\alpha, \beta$ are nilpotent.

2. Appealing to the Jordan Canonical Form gives that, up to similarity, there are only two nonzero nilpotent matrices in $M(3, \Bbb R)$, namely $$\begin{pmatrix}0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0\end{pmatrix} \qquad \text{and} \qquad \begin{pmatrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{pmatrix}.$$