Number of fixed points of generators of reflections (Coxeter) group

Question

Say I have a group with presentation like

$$\langle s,t,u \mid s^2,t^2,u^2,(st)^2,(su)^3,(ut)^4\rangle,$$

faithful on set $S$ with exactly one orbit ($|S|$ is known). How could I determine $|\text{Fix}(s)|$, the number of elements fixed by $s$ (equivalently, how many transpositions produce s)? If necessary, suppose I know $|\text{Fix}((st)^n)|$ for all pairs of the generators and for all $n$.

For example, say |S| = 6, and:

• $|\text{Fix}(st)|$ = 0
• $|\text{Fix}(su)|$ = 0
• $|\text{Fix}(tu)|$ = 2

I know that $st$ must be composed of 3 2-cycles (3 transpositions), $su$ is composed of 2 3-cycles (4 transpositions), and $ut$ is composed of 1 4-cycle (3 transpositions). Since $s$, $t$, and $u$ have order 2, they must be products of disjoint transpositions. Therefore, $$|\text{Transpositions}(s)| + |\text{Transpositions}(t)| \geq 3 $$ $$|\text{Transpositions}(s)| + |\text{Transpositions}(u)| \geq 4 $$ $$|\text{Transpositions}(u)| + |\text{Transpositions}(t)| \geq 3 $$ where ${\rm Transpositions}(\sigma)$ is the number of transpositions needed to produce $\sigma$. If you can solve that then $$|\text{Fix}(s)| = n - 2\cdot|\text{Transpositions}(s)|$$ I doubt those thoughts help but that’s as far as I’ve gotten that I’m sure about.

Answer 1

I don't believe you can deduce this from the information given.

For example, take the vertices of the truncated cube as $S$: https://en.wikipedia.org/wiki/Truncated_cube

It has the symmetry you described, none of the rotations $st$, $su$ or $tu$ fix any of its vertices, but there are certain reflections (6 out of 9) that keep 4 vertices fixed.

If you take $S$ to be (the vertices of) two concentric truncated cubes of different sizes, you would have $|S|=48$ and

• $|\mathrm{Fix}(st)^k|=0$ $(k<2)$
• $|\mathrm{Fix}(su)^k|=0$ $(k<3)$
• $|\mathrm{Fix}(tu)^k|=0$ $(k<4)$

Similarly, if you take $S$ to be (the vertices of) the truncated cuboctahedron: https://en.wikipedia.org/wiki/Truncated_cuboctahedron

You would have exactly the same situation ($|S|=48$ and no vertices fixed by rotations). However, for the truncated cubes you have $|\mathrm{Fix}(s)|=8$ and for the truncated cuboctahdron you have $|\mathrm{Fix}(s)|=0$.

Answer 2

So the question is a little ambiguous as to whether it’s really about this particular example or whether it’s about general problems of this form, illustrated with one particular example. In the latter case, the right answer is probably "go read about $G$-sets and the representation theory of finite groups".

In the former case, the size 6 is so small that it’s quite possible there is a unique answer. There is at least one natural model: the symmetry group of the cube acting on its faces (equiv: octahedron, vertices). It would be very easy to ask a computer to check if there are others, but maybe the size is even small enough to do by hand. In particular: you know $st$ up to conjugacy in $S_6$, so fix a choice, say $st \mapsto (123)(456)$ without loss of generality. We have 15 choices of a fixed point-free involution $su$ (and, incidentally, note that these two elements together will already be transitive necessarily), many of which are the same up to conjugacy by the commutator of the chosen image of $st$. Since $tu = (st)^{-1} \cdot (tu)$, you can easily check whether each possible $su$ image is consistent with the given information (that product should either be a four-cycle or two two-cycles).

Once you've chosen $su$ and $st$, the choice of $s$ determines the whole group. A priori there are $15 + 45 + 15 + 1$ options (all the elements with order dividing $2$), but in fact since $t = s \cdot (st)$ also has to have order dividing $2$, your options for $s$ are super limited: try to convince yourself that the only way you can get a product of two elements of order dividing $2$ can multiply to $(123)(456)$ is if you factor the two three-cycles as a product of two transpositions separately and pair up, so the two factors have cycle type $2^2 1^2$, and there are really only $9$ choices for $s$. Each of these nine choices either does or does not give an element $u = s \cdot (su)$ of order $2$. And then at the end of the day you see what you've got.

(On reflection, if I were really going to do this by hand, I might choose $s$ before $su$ -- having only fixed $st$, we could actually take $s = (12)(34)$ without loss of generality.)