$X$ and $Y$ are $n\times n$ matrices such that rank of $X-Y$ is $1$.

Question

Suppose that $X$ and $Y$ are $n\times n$ complex matrices such that $2Y^2=XY-YX$ and the rank of $X-Y$ is $1$. Prove that $Y^3=YXY$.

I have shown that, trace $Y^n=0$ for all $n=2,3,4,\ldots$. Also we can write $X-Y=xy^t$. From this how to proceed. Please give some idea.

Answer 1

This is true over any field of characteristic zero. Since $X-Y$ has rank one, $E:=X-Y=uv^T$ for some nonzero vectors $u$ and $v$. Therefore $2Y^2=XY-YX=EY-YE$ and \begin{align} 2Y^3=Y(EY-YE)&=(EY-YE)Y,\tag{1}\\ 2YEY&=Y^2E+EY^2.\tag{2} \end{align} Since the commutator $EY-YE$ commutes with $Y$ in $(1)$, by Jacobson's lemma it must be nilpotent. Therefore $2Y^2$ is nilpotent and in turn, $Y$ is nilpotent.

Pick two matrices $U$ and $V$ such that $\ker(U)=\operatorname{span}\{u\}$ and $\ker(V)=\operatorname{span}\{v\}$. From $(2)$, we obtain $UYEYV^T=0$, i.e. $(UYu)(VY^Tv)^T=0$. Hence $u$ is an eigenvector or $v$ is an eigenvector of $Y^T$. But then we must have $Yu=0$ or $Y^Tv=0$, because $Y$ is nilpotent. Thus $YE=0$ or $EY=0$. Consequently $Y(X-Y)Y=YEY=0$, i.e., $Y^3=YXY$.

Actually, since $YEY=0$ and at least one of $YE$ or $EY$ is zero, $(1)$ implies that $Y^3=0$, as observed in loup blanc's answer.

Answer 2

To speak the truth, to show that $Y^3=YXY$, that does not make dream at all.

We can get more interesting results if we take the Omnomnomnom's good idea -cf. his comments-.

$\textbf{Proposition}$. Let $X,Y\in M_n(\mathbb{C})$ s.t. $(*)$ $2Y^2=XY-YX$ and $\mathrm{rank}(X)=1$. Then
i) $YXY=0$
ii) $Y^3=0$.

$\textbf{Proof}$. ​i) is equivalent to the result proved by user1551.

For ii) note that $Y$ is nilpotent (Jacobson) and that $\mathrm{rank}(Y^2)\leq 2$; consequently $Y^4=0$; yet, we can do better.

Remark that $2Y^3=-Y^2X=XY^2$ and $2Y^3=-1/2(XYX-YX^2)$ implies that $Y^2X^2=0$.

There are $2$ cases:

Case 1. $X^2=aX$ where $a\not= 0$. Then $Y^2X=0$ and $Y^3=0$.

Case 2. $X$ is nilpotent ($X^2=0$). We may assume that $X=\mathrm{diag}(J,0_{n-2})$ where $J=\begin{pmatrix}0&1\\0&0\end{pmatrix}$. Let $Y=[y_{i,j}]$.

Note that $2Y^2X=XYX=-4Y^3$ and that $XYX$ has only one non-zero entry $(XYX)_{1,2}=y_{2,1}$. $(XYX)Y=0$ implies that $(XYXY)_{1,1}={y_{2,1}}^2=0$ and, therefore $Y^3=0$.

$\textbf{Remark}$. $(*)$ is a standard equation. Indeed, if we put $X=2A,Y=A+B$ in $(*)$, then we obtain the conditions

$(**)$ $A^2+B^2+2BA=0$ and $\mathrm{rank}(A)=1$.

Of course, this does not result (when $n>2$) in $(A+B)^2=0$, but almost.... Indeed, from Proposition above, we deduce the corollary

$\textbf{Corollary}$. If $A,B$ satisfy $(**)$, then $(A+B)^3=0$.