Let $n$ be an integer greater than or equal to 2 and let $A$ be a $n^{th}$ order complex square matrix. Show that if $A^{n-1}$ is not diagonalizable and $A^n$ is diagonalizable, then $A^n=0$.
I have tried to think of a way to prove that but couldn't. I know that, If a matrix is diagonalizable over a field $F$ if and only if its minimal polynomial is a product of distinct linear factors over $F$. Can we use this to prove it or what is the proper way to think about it?
On
a nice way to do this is to exploit the inequality
(i) $\text{rank}\big(A B\big) \leq \text{rank}\big( A\big)$
this immediately implies
$\text{rank}\big(A^n\big) \leq \text{rank}\big( A^{n-1}\big)\leq ... \leq \text{rank}\big(A^2\big)\leq \text{rank}\big(A\big)$
checking the equality conditions of (i), we see that
$\text{rank}\big(A^k\big) = \text{rank}\big( A^{k-1}\big)$ implies $A$ is injective on the vector space $A^{k-1}\cdot V$, which implies $\text{rank}\big(A^{k+1}\big)= \text{rank}\big(A^k\big) = \text{rank}\big( A^{k-1}\big)$ and so on.
Thus we have 2 cases to consider
1.) $\text{rank}\big(A^n\big) \lt \text{rank}\big( A^{n-1}\big)$
$\implies \text{rank}\big(A^n\big) \lt \text{rank}\big( A^{n-1}\big)\lt ... \lt \text{rank}\big(A^2\big)\lt \text{rank}\big(A\big)\leq n-1$
$\implies \text{rank}\big(A^n\big) = 0 \implies A^n =\mathbf 0$
2.) $\text{rank}\big(A^n\big) = \text{rank}\big( A^{n-1}\big)$
this implies that the eigenvalue 0 is semi-simple for $A^{n-1}$ and $A^n$
$A^{n-1} \text{ defective } \implies A \text{ has some non-zero eigenvalue that is not semi-simple }$
call this eigenvalue $\lambda_1$ and via re-scaling $A$ we can assume WLOG that $\lambda_1=1$. This has algebraic multiplicity $r$.
via any triangularization scheme (Jordan, Schur, or others) we have
$S^{-1} A S = \displaystyle \left[\begin{matrix}I_r + N & * & \\ \mathbf 0 & *\end{matrix}\right]$
and
$S^{-1} A^n S = \displaystyle \left[\begin{matrix}(I_r + N)^n & * & \\ \mathbf 0 & *\end{matrix}\right]=\displaystyle \left[\begin{matrix} I_r & * & \\ \mathbf 0 & *\end{matrix}\right]$
where $N$ is some non-zero nilpotent matrix
(book-keeping note: $N$ is $r\times r$ so argument 1 tells us $N^r=\mathbf 0$)
Thus we have
$I_r = (I_r + N)^n = I_r + \binom{n}{1}N^1+ \binom{n}{2}N^2+....+ \binom{n}{r-1}N^{r-1}$
$\implies \mathbf 0 = \binom{n}{1}N^1+ \binom{n}{2}N^2+....+ \binom{n}{r-1}N^{r-1}$
which shows that these $(r-1)$ powers of $N$ are linearly dependent, a contradiction.
note 1: $N^r=\mathbf 0$ but $N\neq \mathbf 0$ implies there is a min positive integer $k\geq 2$ such that $N^k=\mathbf 0$, multiply the above linear relation by $N^{k-2}$ to verify the contradiction
note 2: This highlights the role of our field being of characteristic zero (i.e. the scalars given by the binomial coefficients are necessarily non-zero). The above argument fails for fields of positive (prime) characteristic -- repeatedly applying the Frobenius Homomorphism reveals the existence of unipotent elements in such a case.
Write $\mathbb{C}^n$ as a direct sum of characteristic spaces $G_{\lambda_i}$ for $A$. Then, for each $G_{\lambda_i}$, $A^n$ is an endomorphism of this space and the $n$-th power of $\lambda_i I+\nu$, for some nilpotent $\nu$, so $\lambda_i^n$ is its only eigenvalue and $A^n_{|G_{\lambda_i}}=\lambda_i^n I$.
If $\lambda_i \neq 0$, then $A_{|G_{\lambda_i}}$ vanishes the polynomial $X^n-\lambda_i^n$ which has simple roots, so $A_{|G_{\lambda_i}}$ is diagonalizable.
It follows that $A^{n-1}_{|G_0}$ is not diagonalizable (in particular nonzero), while $A^n_{|G_0}=0$ by Cayley-Hamilton. Let $d$ be the dimension of $G_0$, $A_{|G_0}$ is nilpotent so $A^d_{|G_0}=0$. So $d > n-1$ thus $d=n$, hence $G_0=\mathbb{C}^n$. Thus $A$ nilpotent and by Cayley-Hamilton $A^n=0$.