Is this an appropriate application of the Orbit-Stabilizer Theorem?

Question

Let $G$ be a group and $H \unlhd G$. Then, $\forall h \in H$: $$[G:C_G(h)]=|O_h|$$ where $C_G(h)$ is the centralizer of $h$ in $G$ and $O_h$ is the conjugacy orbit by $h$ (Orbit-Stabilizer Theorem).

Let's now assume $H$ finite; then: $$\sum_{h \in H}|O_h|=\sum_{O_h \in O}\sum_{h' \in O_h}|O_h|=\sum_{O_h \in O}|O_h|^2$$

($O:=\lbrace O_h, h \in H \rbrace$) and finally: $$\sum_{h \in H}[G:C_G(h)]=\sum_{O_h \in O}|O_h|^2$$

Suppose now that $(G,H)$ is such that $|O|=2$ and $|O_1|^2+|O_2|^2=p$, with $p$ odd prime; then we'd have (Fermat's theorem on sums of two squares): $$\sum_{h \in H}[G:C_G(h)] \equiv 1 \bmod 4 \tag 1$$

Unless I'm mistaken on the above, can someone give examples of such pairs $(G,H)$, for which $(1)$ is expected to hold?

Answer 1

First, note that $\{e\}$ is always a conjugacy class in $H$. Since $H$ is normal, the requirement that $|O| = 2$ means that there is a conjugacy class $K$ such that $H = K\cup \{e\}$, and that $|H| = |K|+1$. Since we want $|K|^2 + 1$ to be prime, this means that $|K|$ is even and $|H|$ is odd.

Now, there are a number of statements that can be made about the restrictions on $(G,H)$:

• $G$ cannot be abelian, otherwise $|K| = 1$ and $2$ is not an odd prime.
• $G$ cannot be a finite $p$-group. If $p$ is odd, its normal subgroups intersect the center of $G$ nontrivially thus $H$ would contain at least $p$ conjugacy classes. Since $|H|$ is odd, $p$ cannot be even.
• $(G,G)$ cannot be a pair, as this would imply that $G$ is a $p$-group.
• $G$ cannot be simple and $H$ must be a proper normal subgroup since $(G,G)$ cannot be pair.
• $G$ cannot be quasisimple, as all proper normal subgroups must lie in the center of $G$ and thus have order two or have at least three conjugacy classes.
• $G$ cannot be a product $G_1\times G_2$ where $G_1$ and $G_2$ are $p_1$- and $p_2$-groups for distinct primes $p_1$ and $p_2$. Assume $(G_1\times G_2,H)$ is a pair, and let $\pi_i:G_1\times G_2\to G_i$ be the projection map for $i=1,2$. Since $|H|$ must have prime power, it must be a subgroup of $G_1\times\{e\}$ or $\{e\}\times G_2$. That means $(G_i, \pi_i(H))$ is a pair for some $i$. This is impossible since $G_i$ are $p$-groups.

Now, let's do some computations in common non-ableian groups and see what examples pop out.

---

Let's take $G = S_n$. Since $H$ is normal, we know that $H = A_n$. However, we know that $|H|$ is odd, which is only true for $A_n$ when $n\le 3$. Since $A_2$ is trivial, $A_3$ is the only possible candidate. It turns out that $(S_3, A_3)$ is a valid pair!

---

Now, let $G$ be the dihedral group

$$D_n = \langle a, x | a^n = x^2 = e, xax = a^{-1}\rangle.$$

Let's assume that $n$ is odd. The only normal subgroups of $D_n$ are then $\langle a^k\rangle$ when $k~|~n$, so $H = \langle a^k\rangle$. Moreover, there are two types of conjugacy classes: all elements not in $\langle a\rangle$, or pairs $\{a^k, a^{-k}\}$. Since the first type isn't a group when we add in $e$, it follows that we need $H = \{e, a^k, a^{-k}\}$. This implies that $a^k$ has order three, so $n = 3k$. Note that this pair $(D_{3k}, \langle a^k\rangle)$ for $n=3$ is exactly the $(S_3, A_3)$ example above.

If $n$ is even, there are two other types of normal subgroups possible: $\langle a^2, x\rangle$ and $\langle a^2, ax\rangle$. However, the conjugacy classes look like pairs $\{a^k, a^{-k}\}$ and the two sets $\{a^{2s}x|s\in\mathbb{Z}\}$, and $\{a^{2s+1}x | s\in\mathbb{Z}\}$. This means that neither $\langle a^2, x\rangle$ nor $\langle a^2, ax\rangle$ can be options for $H$ since they both contain more than two conjugacy classes. Thus, the only option is the same example as above, where $n = 3k$ and $H = \langle a^k\rangle$.

---

I haven't been able to find any examples where $|H|\ne 3$, and it seems like these pairs are going to be fairly rare. I'm curious if there are any nilpotent examples -- these groups are products of $p$-groups, and the last bullet point above makes it seem as though this is impossible.

Answer 2

The identity of $H$ is one conjugacy class. Hence, if $H$ has only two conjugacy classes (in $G$), then every non-trivial element of $H$ must have the same order, which must then be a prime $q$ (conjugate elements have the same order, and $H$ has at least one element of prime order). We deduce that $|H| = q^n$. On the other hand, there are certainly pairs $(G,H)$ with $|H| = q^n$ and such that $H$ consists of two $G$-conjugacy classes. Probably the easiest construction is to take $H$ to be an elementary $q$-group and to use semi-direct products, for example:

$$H = (\mathbf{F}_q)^n, \qquad G = H \rtimes \mathrm{GL}_n(\mathbf{F}_q).$$

If $n = 1$, this is just the Frobenius group $F$ of order $q(q-1)$ which is isomorphic to the normalizer of a $q$-cycle $(1,2,\ldots,q)$ in the symmetric group $S_q$. The centralizer of any $q$-cycle $\sigma$ in $F$ (and even in $S_q$) is just the $q$-cycle itself, so $[F:C_F(\sigma)] = q-1$ for the $q-1$ non-trivial elements, and of course $[F:C_F(1)] = 1$.

Your final requirement is that

$$p = 1 + (q^n - 1)^2$$

is prime. Standard conjectures imply the existence of infinitely many pairs of primes of this form (even with $n = 1$), and it is easy enough to write down many examples, e.g.

$$5 = 1 + (3-1)^2, \ 17 = 1 + (5-1)^2, 37 = 1 + (7-1)^2, \ 101 = 1 + (11 - 1)^2$$ $$257 = 1 + (17-1)^2, \ 577 = 1 + (5^2 - 1)^2, \ldots $$