${\bf Problem }$ If $A$ is a square matrix, we define the trace of A, $tr(A)$, to be the sum of the diagonal entries of $A$. Let V be a finite dimensional vector space over $\Bbb C$ , with $\dim(V ) = n \geq 1$, and let $T \in L(V )$. We define the trace of T by $tr(T) = tr([T]_{\alpha}^{\alpha})$ where $\alpha $ is any basis for V .
${\bf Part (a)}$
Prove that $tr(T)$ is well-defined. That is, prove that if $\alpha $ and $\beta$ are bases for V , then $tr([T]_\alpha^\alpha) = tr([T]_\beta^\beta) $
Note that a basis transformation of a transformation $T$ is $ATA^{-1}$ where $A$ is an invertible matrix, we also know that the trace has the property $\operatorname{tr}(AB)=\operatorname{tr}(BA)$. This implies that $$\operatorname{tr}((AT)A^{-1}) = \operatorname{tr}(A^{-1}(AT))= \operatorname{tr}(A^{-1}AT) = \operatorname{tr}T$$ the result follows.
${\bf Part (b)}$
Prove that T is nilpotent iff $tr(T) = tr ( T^2) = \cdots = tr(T^n)=0 $
We start by showing that If $T\in L(V) $ is nilpotent then we prove that the only eigenvalue of T is $0$. Then there is a positive integer $n$ s.t $T^n =0$
This implies that T is not injective hence $0$ is an eigenvalue of T.
Conversely, suppose $\lambda $ is an eigenvalue of T.
Then there exists a non-zero vector $v\in V $ such that $\lambda v = Tv $ we apply T $n$ times and we get that $\lambda^n v = T^nv $ but $ T^n v=0 $ it follows that $ \lambda^n v =0 $ but v is non-zero so $ \lambda =0$
Secondly we have that the characteristic polynomial of an $n\times n$ matrix $A$ is equivalent to $$p(t) = \det(A-tI) = (-1)^n \big(t^n - (\text{tr} A) \,t^{n-1} + \dots + (-1)^n \det A\big)\,.$$ On the other hand, $p(t) = (-1)^n(t-\lambda_1)\dots (t-\lambda_n)$, where the $\lambda_j$ are the eigenvalues of $A$. So, comparing coefficients, we have $\text{tr}A = \lambda_1 + \dots + \lambda_n$.
Now $( \Rightarrow ) $ assume that T is nilpotent we know by above that the only eigenvalues of T are zero and also by above that the $\operatorname{tr}(T) =0 $ now we wish to show that if all the eigenvalues are $0$ of T that all the eigenvalues of $T^k $ are zero for any $k \in \Bbb N$ well we know by part (a) that the trace is independent of the basis and we also know we can always find a basis so that the matrix T is upper triangular with all of the diagonal entries also equal to zero by above. Now consider $\operatorname{tr} T^k$ where k is any integer we know that $\operatorname{tr} (T^k)=\operatorname{tr} (ATA^{-1})^k =\operatorname{tr} (U^k) $ where A is a change of basis matrix to make T upper triangular which we denote as U. we notice that $\operatorname{tr} (U^k)=\operatorname{tr} (U (U^{k-1}) $ Note that $U^{k-1} $ is the product of $k-1$ upper triangular matrices so it is upper triangular let us denote it $R $ let $C=UR $ but all the diagonal entries of $U $ are zero we have that the $C_{ij} = \sum_{k =1}^n a_{ik}b_{ki} = \sum_{k =1}^{i-1}a_{ik}b_{kj} + \sum_{k=i}^na_{ki}b_{kj} = 0 + 0 = 0, $
Since $k \leq i \implies a_{ik} =0 $ and $k \geq i \implies b_{kj} =0 $ for this particular combination of $(i,j)$. We conclude that $C_{ij} = 0$ if $i \geq j$ hence $C$ is upper triangular and all of its diagonal entries are zero. it follows that $\operatorname {tr}(C) =0 = \operatorname{tr} (U^k) = \operatorname{tr} (T^k)$ for any value of k so the result follows.
Now for the hard part:
$( \Leftarrow ) $ Assume that $tr(T) = tr ( T^2) = \cdots = tr(T^n)=0 $ want to show that T is nilpotent.
Let $ m_i$ denote the multiplicity of the eigenvalue and let $\lambda_1, \cdots , \lambda_r $ denote all the distinct nonzero eigenvalues. using an above result and the assumption we have that $$ m_1 \lambda_1 + \cdots + m_r \lambda_r =0 $$ $$ m_1 (\lambda_1)^2 + \cdots + m_r (\lambda_r)^2 =0 $$ $$ \vdots $$
$$ m_1 (\lambda_1)^n + \cdots + m_r (\lambda_r)^n =0 $$
We have that $$\begin{pmatrix} \lambda_1 & \cdots & \lambda_r \\ (\lambda_1)^2 & \cdots & (\lambda_r)^2 \\ \vdots & \ddots & \vdots \\ (\lambda_1)^{n} & \cdots & (\lambda_r)^{n} \end{pmatrix} \begin{pmatrix} m_1 \\ \vdots \\ m_r \end{pmatrix} = \begin{pmatrix} 0 \\ \vdots \\ 0 \end{pmatrix}$$
If we can conclude that each $ m_i=0 $ then the result will follow and it is sort of clear That this sucker is non-invertable (i think but not sure on the justification) so we know the determinant is zero. We are also given that a Vandermonde-matrix is a matrix of this form:
$$\begin{pmatrix} 1 & \cdots & 1 \\ a_1 & \cdots & a_r \\ \vdots & \ddots & \vdots \\ a_1^{r-1} & \cdots & a_r^{r-1} \end{pmatrix} $$ where each $a_i \in \Bbb C$ and $\det(V)= \Pi_{1\leq i j \leq r} (a_j - a_i)$ we are given this as a true fact to use w.o proof and it seem to be the key to finishing off the problem. ( i did try and prove the required result with induction my only suggestion on that avenue is don't.)
I propose another method:
If $A$ is nilpotent, then $A$ is triangulable over $\mathbb C$ with $0$ as unique eigenvalue: $A = P^{-1}TP$. Hence $A^k = P^{-1}T^kP$ with $T^k$, as $T$, a triangular matrix with $0$ on diagonal. Hence $Tr(A^k) = 0$.
If $Tr(A) = ... = Tr(A^n) = 0$ then for all polynomial $P$ with degree $≤ n$ and such that $P(0) = 0$, $Tr(P(A)) = 0$ (by linearity). $A$ is triangulable over $\mathbb C$: $T = Q^{-1}AQ$.
Take $P$ with degree $≤ n$ such that $P(0) = 0$ and for all eigenvalue $\lambda \neq 0$ of $A$, $P(\lambda) = 1$ (Lagrange polynomial). Then $Tr(P(T)) = Tr(P(A)) = 0$ implies all eigenvalue is $0$ ; so $A$ is nilpotent.