Let $T:V \to V$ be a linear operator such that $T^5 \neq 0,$ but $T^6 = 0.$ Suppose $V$ is isomorphic to $\mathbb{R}^6.$ Prove that there does not exist an $S:V \to V$ such that $S^2 = T.$ Does the answer change if $V$ is isomorphic to $\mathbb{R}^{12}?$
My thoughts for the first part: Suppose such an $S$ did exist, then $T^6 = S^{12} = 0.$ but if $S^{12} = 0,$ then $S^{6} = 0,$ which implies that $T^3 = 0$ and therefore $T^5 = 0,$ contradicting the given information.
I am just starting with these type of problems so I am not sure how to formalize these ideas, but I have a feeling that theorems involving nilpotence should be used. If anyone has a suggestion on how to make this argument more rigorous, it would be of much help.
For the second part, $S^{12} = 0$ does not imply any lower power of $S$ would be the zero matrix, so then an $S$ could exist such that $S^2=T,$ or $S^10 = T^5,$ while still satisfying $T^6 = S^{12} = 0.$
Any suggestions on how to formalize this (or better ways to do it) would be appreciated.
Your argument is mainly correct, but you ought to make rigorous the argument that $S^{12} = 0$ implies $S^{6} = 0$. The way that I know of to do this is using the minimal polynomial: $S$ is a root of $f(X) = X^{12}$, so the minimal polynomial of $S$ divides $f(X)$. Thus, the minimal polynomial is $X^{n}$ for some $n$; but $n \leqslant 6$, since the minimal polynomial of $S$ divides the characteristic polynomial of $S$ by Cayley-Hamilton.
You are also correct that the above argument does not generalize if $V \cong \mathbb{R}^{12}$. Indeed, here is an explicit counterexample: take $S$ to be the matrix whose entries are all zero except ones along the super diagonal. Then $S^{12} = 0$, and this is the minimal polynomial of $S$; taking $T = S^{2}$ provides the desired example.