For an invertible matrix $P\in\mathbb{R}^{n\times n}$ let $T_P\colon\mathbb{R}^{n\times n}\to\mathbb{R}^{n\times n}$ be the linear map defined by $T_P(M)=PMP^{-1}$ for any $M\in\mathbb{R}^{n\times n}$.
Let $O$ be the $3\times3$ rotation matrix $$O=\begin{pmatrix}0&-1&0\\1&0&0\\0&0&1\end{pmatrix}$$ Compute the minimal and characteristic polynomials of $T_O$ on $S(3)$, where $S(n)\subset\mathbb{R}^{n\times n}$ is the space of symmetric $n\times n$ matrices.
Hint: Note that $O^4=I$.
I would appreciate a solution or at least a hint of how to approach this problem in a smart way using the given hint. Unfortunately all I can think is a brute-force solution where one would write down the matrix of $T_O$ in a basis of $S(3)$, then compute the characteristic polynomial by definition via determinant and try to guess the minimal polynomial as a factor of the already known characteristic polynomial. Although doable, it seems to be boring as hell because of $\mbox{dim}S(3)=6$ and the resulting sequence of $2\cdot6=12$ multiplications of $3\times3$ matrices. On the other hand, $O^4=I$ seems like a shortcut to finding the polynomial which annihilates $T_O$. So how can I do it? Thanks a lot in advance.
Consider the minimal polynomial $\mu(x)$ and the characteristic polynomial $\chi(x)$ of $T_O.$ Piggybacking off of @Azif00's comment above, we have that $\mu(x)$ divides $p(x) = x^4 - 1 = (x - 1)(x + 1)(x^2 + 1).$ Recall that the Smith Normal Form of $xI - T_O$ is the $3 \times 3$ diagonal matrix whose diagonal entries are some nonzero polynomials $f_1(x),$ $f_2(x),$ and $f_3(x)$ in $\mathbb R[x]$ that satisfy $f_1(x) \,|\, f_2(x)$ and $f_2(x) \,|\, f_3(x)$ with $\mu(x) = f_3(x)$ and $\chi(x) = f_1(x) f_2(x) f_3(x).$ (We refer to the $f_i(x)$ as the invariant factors of $T_O.$)
Certainly, we cannot have that $\mu(x) = x^2 + 1,$ as this is irreducible over $\mathbb R,$ hence we must have that $(x - 1) \,|\, \mu(x)$ or $(x + 1) \,|\, \mu(x).$ Further, no higher power of these linear polynomials can divide $\mu(x)$ -- else $\mu(x)$ would not divide $p(x)$ -- and these polynomials are relatively prime. But in view of the fact that $\deg \chi = \dim_\mathbb R S(3) = 6,$ we must also have that $\deg f_1 + \deg f_2 \geq 2,$ from which it follows that we have four possibilities.
1.) $\mu(x) = p(x)$ and $\chi(x) = (x - 1)^2(x + 1)^2 (x^2 + 1)$
2.) $\mu(x) = (x + 1)(x^2 + 1)$ and $\chi(x) = (x + 1)^2 (x^2 + 1)^2$
3.) $\mu(x) = (x - 1)(x^2 + 1)$ and $\chi(x) = (x - 1)^2 (x^2 + 1)^2$
4.) $\mu(x) = (x - 1)(x + 1)$ and $\chi(x) = (x - 1)^3 (x + 1)^3$
Considering that $T_O(I) = OIO^{-1} = I,$ it follows that $1$ is an eigenvalue of $T_O,$ hence the polynomial $(x - 1)$ divides $\chi(x);$ that narrows our options down to (1.), (3.), or (4.). Can you determine if $-1$ is also an eigenvalue of $T_O?$ I don't see any obvious matrix candidates that satisfy $T_O(M) = -M,$ but if we could find something, we could limit the options to (1.) and (4.).
Ultimately, it seems that this method of computing $\mu(x)$ and $\chi(x)$ is not super fruitful in this case; however, there are some times when it can be used to great effect.