What's the classification of Coxeter Groups for which every proper parabolic subgroup is finite?

Question

Let $(W,S)$ be a Coxeter Group. I want to know exactly which Coxeter Groups have the property $\forall J \subsetneqq S $, $W_J$ is finite.

I can think of the finite Coxeter Groups, the Affine Coxeter Groups. A few more quickly come to mind. Here's some bad drawings meant to represent their Coxeter diagrams:

o ----4---- o ----5---- o

o ----5---- o ----3---- o ----3---- o ----5---- o

I could think of a few more and could imagine trying to inductively and exhaustively find all such examples.

I believe these are both (compact?) hyperbolic subgroups but know little about the intimate details of this area. Is it true that all Coxeter Groups with finite proper parabolic subgroups is either Finite, Affine or Compact Hyperbolic? And if so does the compact Hyperbolic Coxeter Groups have a nice classification?

I've searched for it online but not found a unifying answer easily.

Answer 1

Consider the following result, which appears as Proposition 6.8 in Humphreys (with further comments on p 140), and Exercise 14 in chapter V of Bourbaki.

Theorem: Let $(W,S)$ be an irreducible Coxeter system, with graph $\Gamma$ and associated bilinear form $B$. It is compact hyperbolic if and only if the following conditions are satisfied:

1. $B$ is non-degenerate, but not positive definite.
2. For each $s\in S$, the Coxeter graph obtained by removing $s$ from $\Gamma$ is of finite type (ie the corresponding special subgroup is finite).

Here compact hyperbolic means that the standard geometric representation $(W,S)$ on $\mathbb{R}^n$ induces an action on hyperbolic space modelled as one component of $\{\lambda\in \mathbb{R}^n\mid B(\lambda,\lambda)=-1\}$ which is discrete and cocompact.

Now is an appropriate point to mention that the (compact) hyperbolic groups have indeed been classified, and that classification can be found in $\S$6.9 of Humphreys or exercise 15 of Bourbaki. I have also included them all here, note there are finitely many, and only in 3, 4, and 5 dimensions.

If you are happy for your special subgroups to be positive semi-definite, rather than positive definite, then replace compact hyperbolic in the above theorem with hyperbolic (cf Humphreys Proposition 6.8 and exercise 13 in Bourbaki).

Clearly as you say, if $(W,S)$ is finite and irreducible, then all its special subgroups are also finite, so the only remaining case to consider is when $B$ is degenerate. For this case we can appeal to Proposition V.4.10 in Bourbaki which essentially says the following

Theorem: Let $(W,S)$ be a Coxeter system, with graph $\Gamma$ and associated bilinear form $B$. If $B$ is positive and degenerate then $W$ acts discretely by orthogonal reflections on some Euclidean space.

The classification of these irreducible affine Coxeter groups is well-known and each satisfies the condition you want on special subgroups.

The only case not covered is if $B$ is degenerate and not positive. I am not familiar with any results on this class of Coxeter groups I am afraid. You may be reduced to considering all possible extensions of finite type Coxeter diagrams. This probably wouldn't be quite as bad as it first seems because the proof of the classification of finite Coxeter groups gives very tight conditions on the local structure of their diagrams which would imply something like the following:

• Each new vertex connects to $\Gamma$ by at most 2 edges (since finite type diagrams don't contain any cycles)
• If there are two connecting edges, these must connect to leaves of $\Gamma$ (for the same reason)
• No connecting has a label greater than 5 (because if a finite type diagram has a label greater than 5 it is dihedral)
• If the connecting edge has label greater than 3, it connects to a leaf of $\Gamma$
• No connecting edge connects to a vertex of $\Gamma$ having degree greater than 3 unless perhaps $\Gamma=D_4$ (since no finite type diagram has a vertex of degree greater than 3)

And possibly a few others, so actually the possibilities are very limited.