Let $M_n$ be the vector space of $n \times n$ real matrices.
We say a linear operator $\alpha:M_n \to M_n$ is hemitropic* if:
$(*) \, \, \alpha(S^TXS)=S^T\alpha(X)S \, , \, \forall S \in SO(n)$
and isotropic, if the above formula holds $\forall S \in O(n)$
My question: Is every hemitropic operator necessarily isotropic? Does the answer change if we assume $\alpha$ is injective?
In odd dimensions, the answer is yes, since for any $Q \in O(n) \setminus SO(n)$, $\det(-Q)=1$ so $-Q \in SO(n)$ and by (*):
$\alpha(Q^TXQ)=\alpha\big((-Q)^TX(-Q)\big)=(-Q)^T\alpha(X)(-Q)=Q^T\alpha(X)Q$
I suspect the answer in even dimensions is negative, but so far I didn't find a way to construct a hemitropic non-isotropic operator. (Though I guess this can be done even in dimension $2$).
Update:
For dimension $2$ I have constructed (see answer below) a hemitropic non-isotropic operator. However, my construction relied on the fact that all $2$-dimensional rotation commute. This is not the case for higher dimensions. (See here).
This leaves open the question for even dimensions above two.
The terminology "hemitropic" comes from Elasticity theory.
$\newcommand{\al}{\alpha}$ $\newcommand{\be}{\beta}$ $\newcommand{\the}{\theta}$
Well, it turns out the answer is indeed negative.
Even for dimension $2$, there are hemiropic, non-isotropic operators. (In particular injective ones).
Here is an example:
$\alpha:M_2 \to M_2, \al(A)=R_{\theta} \cdot A$ where $R_{\theta}=\begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{pmatrix}$ is a rotation matrix.
$\al$ is linear and invertible.
We show $\al$ is hemitropic ($SO(n)$-invariant):
This is essentially the fact that any two $2$-dimensonal rotations commute.
$SO(2)=\{R_{\be}| \be \in [0,2\pi) \} \, , \, R_{\be}^T=R_{\be}^{-1}=R_{-\be}$
$\al(R_{\be}^TAR_{\be})=\al(R_{-\be}AR_{\be})=R_{\the}R_{-\be}AR_{\be}\stackrel{(*)}{=} R_{-\be}R_{\the}AR_{\be}=R_{\be}^T\al(A)R_{\be}$
as required. (Where (*) follows from the fact that any two rotations commute).
We show $\al$ is not isotropic ($O(n)$-invariant):
This is essentially because two $2$-dimensonal rotations do not commute with reflections.
Denote $R_{\theta}=\begin{pmatrix} q & -s \\ s & q \end{pmatrix} \, , q=\cos(\the),s=\sin(\the)$
Choose $A=\begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, O=O^T=\begin{pmatrix} -1 & 0 \\ 0 & 1 \end{pmatrix} \in O(n) \setminus SO(n)$
$\al(A)=\begin{pmatrix} q & -s \\ s & q \end{pmatrix} \cdot \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} -s & 0 \\ q & 0 \end{pmatrix}$
In general: $O \cdot \begin{pmatrix} a & b \\ c & d \end{pmatrix} \cdot O = \begin{pmatrix} a & -b \\ -c & d \end{pmatrix}$, so:
$(1): \, \, O \al(A) O = O \begin{pmatrix} -s & 0 \\ q & 0 \end{pmatrix} O=\begin{pmatrix} -s & 0 \\ -q & 0 \end{pmatrix}$, but
$(2): \, \, \al(OAO)=\al(\begin{pmatrix} 0 & 0 \\ -1 & 0 \end{pmatrix})=\begin{pmatrix} q & -s \\ s & q \end{pmatrix} \cdot \begin{pmatrix} 0 & 0 \\ -1 & 0 \end{pmatrix} =\begin{pmatrix} s & 0 \\ -q & 0 \end{pmatrix}$
So, $(1),(2) \Rightarrow O \al(A) O \neq \al(OAO)$ so $\al$ is not isotropic, as required.