Non-Abelian subgroups and invariants in a unitary group 2

Question

Related to a previous question, let us redefine their $\{P_1, P_2, P_3, -P_1, - P_2, - P_3 \}$ to something somehow different.

Let $$G=U(3),$$ be the unitary group. Here we consider $G$ in terms of the fundamental representation of U(3). Namely, all of $g \in G$ can be written as a rank-3 (3 by 3) matrices.

1. Can we find some subgroup of Lie group, $$k \in K \subset G= U(3) $$ such that

   $$ k^T \{P_1, P_2, P_3, -P_1, - P_2, - P_3 \} k =\{P_1, P_2, P_3, -P_1, - P_2, - P_3\}. $$ This means that set $\{P_1, P_2, P_3, -P_1, - P_2, - P_3\}$ is invariant under the transformation by $k$. Here $k^T$ is the transpose of $k$. What is the full subset (or subgroup) of $K$?

Here we define: $$ P_1 = \left( \begin{array}{ccc} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right),\;\;\;\; P_2 = \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \\ \end{array} \right),\;\;\;\; P_3 = \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \\ \end{array} \right).$$

This means that $k^T P_a k= \pm P_b$ which may transform $a$ to a different value $b$, where $a,b \in \{1,2,3 \}$. But overall the full set $ \{P_1, P_2, P_3, -P_1, - P_2, - P_3\}$ is invariant under the transformation by $k$.

There must be a trivial element $k=$ the rank-3 identity matrix. But what else can it allow?

How could we determine the complete $K$?

Answer 1

The answer (and the method) is the same as the previous question.

Answer. The subgroup $K$ of $U(3)$ containing invariant matrices are isomorphic to the finite group $$ \mathbb{Z}_4\times S_4 \cong\langle i\rangle\times D(2,3,4) $$ where $\langle i\rangle=\{\pm I,\pm iI\}\cong\mathbb{Z}_4$ and $D(2,3,4)$ is the von Dyck group which is isomorphic to $S_4$.

More specifically, $D(2,3,4)=\langle a,b,c \mid a^2=b^3=c^4=abc=I\rangle$ is represented in $U(3)$ as follows: $$ a = \begin{pmatrix} -1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix}, \quad b = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}, \quad c = \begin{pmatrix} 0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

Here is more detailed calculation:

Claim 1. The matrices in $U(3)$ satisfying $k^TP_1k=\pm P_1$ are $$ \begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \quad\text{or}\quad \begin{pmatrix} 0 & \alpha & 0 \\ \beta & 0 & 0 \\ 0 & 0 & \gamma \end{pmatrix} $$ where $|\alpha|=|\beta|=|\gamma|=1$ and $\alpha\beta=\pm1$.

Proof. Let us find $k=(k_{ij})$ such that $k^TP_1k=\pm P_1$. It implies the following equations $$ \begin{gather*} k_{11}k_{21} = k_{12}k_{22} = k_{13}k_{23} = 0, \\ k_{11}k_{22} + k_{12}k_{21} = \pm1, \tag{*} \\ k_{11}k_{23} + k_{13}k_{21} = k_{12}k_{23} + k_{13}k_{22} = 0. \end{gather*} $$ Note that $k_{13}=0$. Otherwise $k_{21}=k_{22}=k_{23}=0$, and it contradicts to $k\in U(3)$. Similarly, $k_{23}=0$. Moreover, $k_{31}=k_{32}=0$ since $k\in U(3)$. Now the remaining equations in (*) $$ k_{11}k_{21} = k_{12}k_{22} = 0, \quad k_{11}k_{22} + k_{12}k_{21} = \pm1 $$ give the matrices in the claim depending on $k_{11}\neq0$ or $0$.

Fix a permutation matrix $Q=\begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}$. Then we have $$ QP_1Q^T=P_2 \quad\text{and}\quad Q^2P_1(Q^2)^T=Q^TP_1Q=P_3 $$

In a similar way as the previous question, we have $$ \begin{align*} K_{00} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_1\} &= K_{00} \\ K_{02} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_2\} &= K_{00}Q^T \\ K_{01} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_3\} &= K_{00}Q \\ K_{10} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_1\} &= QK_{00} \\ K_{12} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_2\} &= QK_{00}Q^T \\ K_{11} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_3\} &= QK_{00}Q \\ K_{20} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_1\} &= Q^TK_{00} \\ K_{22} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_2\} &= Q^TK_{00}Q^T \\ K_{21} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_3\} &= Q^TK_{00}Q \end{align*} $$

Finally, we have the following 16×6 invariant matrices:

Claim 2. For $\alpha,\beta,\gamma\in\mathbb{C}$ satisfying $\alpha\beta=\pm1$, $\alpha\gamma=\pm1$, and $\beta\gamma=\pm1$, $$ \begin{align*} K_{00}\cap K_{12}\cap K_{21} &\Rightarrow \begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \quad K_{00}\cap K_{11}\cap K_{22} \Rightarrow \begin{pmatrix} 0 & \alpha & 0 \\ \beta & 0 & 0 \\ 0 & 0 & \gamma \end{pmatrix} \\ K_{02}\cap K_{10}\cap K_{21} &\Rightarrow\begin{pmatrix} \alpha & 0 & 0 \\ 0 & 0 & \beta \\ 0 & \gamma & 0 \end{pmatrix} \quad K_{02}\cap K_{11}\cap K_{20} \Rightarrow\begin{pmatrix} 0 & 0 & \alpha \\ \beta & 0 & 0 \\ 0 & \gamma & 0 \end{pmatrix} \\ K_{01}\cap K_{10}\cap K_{22} &\Rightarrow\begin{pmatrix} 0 & \alpha & 0 \\ 0 & 0 & \beta \\ \gamma & 0 & 0 \end{pmatrix} \quad K_{01}\cap K_{12}\cap K_{20} \Rightarrow\begin{pmatrix} 0 & 0 & \alpha \\ 0 & \beta & 0 \\ \gamma & 0 & 0 \end{pmatrix} \end{align*} $$