I numerically verified the case for the $2\times2$ matrix but don't know how to prove it theoretically. Let's say, if $\Sigma$ is an $n\times n$ matrix, and we have $A \Sigma A^{\dagger}=\Sigma$, where $A$ is a unitary matrix, and $\Sigma$ is a Hermitian matrix, then does it imply $\Sigma$ also to be a diagonal matrix?
For clarification, $\Sigma$ is real in my case, but I don't know if 'real' really matters to prove $\Sigma$ is diagonal.
Here is a simple numerical verification for $2\times2$ case: Let
$$A = \begin{bmatrix} \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{bmatrix}, $$
which is a rotation matrix, let $\Sigma = \begin{bmatrix} a & b\\ c & d \end{bmatrix}$, from the equation $A \Sigma A^{\dagger}=\Sigma$ we get $a=d, c=-b$, now hermitian implies $c=b$, so we get $c=b=0$ and prove $\Sigma$ is a diagonal matrix.
Your initial conjecture is trivially false, as Mason's counterexample $A=I$ shows.
Therefore, so is your "numerical verification for 2*2 case" (take $\theta=0$).
An answer to (a strengthening of) your modified question (in the comments) is the following: if an arbitrary (not necessarily hermitian) matrix $\Sigma\in M_n(\Bbb C)$ commutes with every unitary matrix $A\in U_n(\Bbb C),$ then $\Sigma$ is a scalar matrix. This is because every complex $n\times n$ matrix is a linear combination of two unitary matrices, and if $\Sigma$ commutes with every complex $n\times n$ matrix, then it is scalar.