Simple subgroup of $S_{n+1}$ of index k

Question

Some help regarding the following problem will be greatly appreciated.

Let $n\in \mathbb{Z}^+$ with $n\geq5$, and suppose that $G$ is a simple subgroup of $S_{n+1}$ of index $k$. Prove that if $k\leq2n+2$, then ($k=2$ and $G=A_{n+1}$) or ($k=2n+2$ and $G\cong A_n$).

Relevant information/What I have so far:

1. Theorem: Let $X$ be a subgroup of $S_n$. Then either $X \leq A_n$, or $X \cap A_n$ is a normal subgroup of $G$ of index 2.

2. $A_n$ is simple for $n\geq 5$

From the above information, we can deduce that $G$ is a subgroup of $A_{n+1}$. Otherwise, $G\cap A_{n+1}$ will be a non-trivial normal subgroup of $G$ which contradicts the fact that $G$ is a simple group.

Another thing that I managed to deduce is that

$[S_{n+1}:A_{n+1}][A_{n+1}:G]=[S_{n+1}:G]=k\leq2n+2$

$\implies2[A_{n+1}:G]=[S_{n+1}:G]=k\leq2n+2$

$\implies[A_{n+1}:G]=k/2\leq n+1$

This means that $G$ cannot be isomorphic to $A_i$, where $i=1,2,...,n-1$.

This is where I got stucked at. It is clear from what I have worked out that ($k=2$ and $G=A_{n+1}$) or ($k=2n+2$ and $G\cong A_n$) would satisfy the given condition. But I do not know how to prove that either one of this two situations has to occur.

Any hints/tips on how to progress would be greatly appreciated.

Answer 1

The case $k=2$ is easy. We must have $G=A_{n+1}$. So w.l.o.g. we can assume that $k>2$. You already saw that $G$ must be a subgroup of $A_{n+1}$ of index $k/2$.

Let $X=A_{n+1}/G$ be the set of left cosets of $G$ in $A_{n+1}$. The set $X$ has $k/2$ elements. This gives rise to a non-trivial homomorphism $\pi:A_{n+1}\to Sym(X)$. The simplicity of $A_{n+1}$ then dictates that $\pi$ must be injective. This immediately forces $k/2\ge n+1$ for otherwise $Sym(X)\simeq S_{k/2}$ is too small to contain a copy of $A_{n+1}$. At this point we thus know that $k=2(n+1)$.

We can also conclude that the restriction of $\pi$ to the simple subgroup $G$ is also injective. But $\pi(G)$ has a fixed point in $X$, namely the coset $1G=G$. Therefore $\pi$ yields a homomorphism $\tilde{\pi}:G\to Sym(X\setminus\{1G\})\simeq S_n$. Repeating the early steps this forces $G\simeq\tilde{\pi}(G)\simeq A_n$.