Subsemigroup of S_n(symmetric group) of largest size

Question

For $n\geq 3$, Is $A_n$ (the alternating group) is the proper subsemigroup of largest size of $S_n$ (the symmetric group of degree $n$)?

Answer 1

Let's assume $n>2$, because $S_2$ has just one proper subgroup.

A subsemigroup $X$ of a finite group $G$ is a subgroup: for each $g\in X$ the map $f_g\colon X\to X$ defined by $f_g(x)=gx$ is injective, hence surjective. Therefore there is $y\in X$ such that $gy=g$, so $y=1\in X$; next there is $z\in X$ such that $gz=1$ and so $z=g^{-1}\in X$.

The subgroup $A_n$ has index $2$ in $S_n$ and $2$ is the least divisor $(>1)$ of $|S_n|=n!$, so $A_n$ is a proper subgroup having the most possible elements. Suppose $H$ is another subgroup of $S_n$ having index $2$; then $H$ is normal in $S_n$ and so $H\cap A_n$ is normal in $A_n$; but $A_n$ is simple when $n>4$ and, in this case, we're left with $H\cap A_n=\{1\}$ or $H\cap A_n=A_n$.

If $H\cap A_n=A_n$, then $A_n\subseteq H$ and so $A_n=H$. If $H\cap A_n=\{1\}$ then $HA_n=S_n$, so $S_n/A_n\cong H/(H\cap A_n)\cong H$ implies $|H|=2$, so that $|S_n|=[S_n:H]\,|H|=4$, a contradiction.

We're left with $n=3$ and $n=4$. The subgroups of $S_3$ are easily classified and $A_3$ is the only subgroup of order $3$ and index $2$. We can dismiss as before the cases $H\cap A_n=\{1\}$ and $H\cap A_n=A_n$. So this intersection can have orders $2,3,4,6$. Since $H\ne A_n$, we have $HA_n=S_n$, so, as before, $$ S_n/H\cong A_n/(A_n\cap H) $$ and so $|A_n\cap H|=6$. Classifying the subgroups of $S_4$ having six elements is easy and again we can exclude that $H\ne A_n$. Of course, more powerful group theoretic tools could be used.