I tried to prove to myself the invariance of $$ (X_{11} - X_{22})^2 + (X_{12}+X_{21})^2 $$ for a 2D tensor $X$ under rotation.
I managed to do it by showing that $$ \left( [ R X R^{-1} ]_{11} - [ R X R^{-1} ]_{22} \right)^2 + \left( [ R X R^{-1} ]_{12} + [ R X R^{-1} ]_{21} \right)^2 = (X_{11} - X_{22})^2 + (X_{12}+X_{21})^2 $$ when $$ R = \left( \begin{matrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{matrix} \right) \mathrm{.} $$ However this was very tedious as I just expanded each $[ R X R^{-1} ]_{ij}$ and essentially used a brute force strategy.
Is there some easier way to show this? The invariance of the trace of $X$ is much nicer to prove.
EDIT: removed the need for $X$ to be symmetric. Has anyone seen this type of invariant anywhere? Might there be a similar one for 3D tensors?
So thanks to the user quarague I found a more elegant solution.
The invariant I want can be written as $$ \mathrm{tr}\left( X^\mathrm{T} X \right) - 2 \det X $$ and I only need to prove the invariance of both these terms (as they actually both are invariant).
This should also work in the 3D case I was interested in.