How do i prove that any subgroup of $A_5$ has order at most 12?

Question

I know this question has been answered Other proofs that subgroups of $A_5$ have order at most 12

But i have difficulty in understanding that proof.The book says we can assume that $A_5$ has no normal subgroup.How to find the proof using this property ?

EDIT- (I solved the previous question ( $A_5$ has no normal subgroup ) but i am not able to solve this problem ).

I have already mentioned that my question similar to Other proofs that subgroups of $A_5$ have order at most 12 so please don't mark it as duplicate.

The book which i am talking about is - Topics In Algerbra by Herstein. (2.10.15)

I am thankful if some can explain the same proof ( provided in link ).I have difficulty in understanding the homomorphic part of that answer.

Answer 1

Let $\;H\;$ be a subgroup of $\;A_5\;$ of order $\;>12\implies\;$ its index is $\;l<5\;$ . Then $\;A_5\;$ acts on the set of left cosets of $\;H\;$ in $\;G\;$ , and this determines a homomorphism $\;\phi:A_5\to S_l\;$ . Since $\;A_5\;$ is simple this homomorphis is acutaly a monomorphism (i.e., $\;1-1$) , and thus it is an injection, which of course is impossible as $\;|A_5|=60>S_l\;$

Answer 2

$A_5$ has order 60. Thus, any proper subgroup $H$ of order larger than 12 must have order 15, 20, or 30. Any subgroup of order 30 would have index 2, so would be normal, contradicting your previous result. Thus, $H$ must have order 15 or 20.

Now, both 15 and 20 are multiples of 5, so $H$ must contain an element of order 5. Up to conjugation, that element is $(1,2,3,4,5)$. If $H$ has order 15, then it also contains an element of order $3$, which must be a 3-cycle. But running through each possible 3-cycle:

$(1,2,3)(1,2,3,4,5)^2 = (1,4)(2,5)$, which has order 2, so $H$ has even order, a contradiction.
$(1,2,4)(1,2,3,4,5)^4 = (2,3)(4,5)$, so again, $H$ has even order. $(1,3,5)(1,2,3,4,5)^4 = (1,2)(3,4)$, so again, $H$ has even order.

And all others are conjugate (as a whole) to those, so symmetrically, have the same orders. Thus, there is no subgroup of order 15.

Thus, $H$, if it exists, must have order 20, so must contain an element of order 2, which must be a product of two disjoint 2-cycles. Again, running through the possibilities:

$(1,2)(3,4)(1,2,3,4,5) = (1,3,5)$, so $H$ has order 60.
$(1,3)(2,4)(1,2,3,4,5)^2 = (1,5,2)$, so $H$ has order 60.
$(1,3)(2,5)(1,2,3,4,5)^2 = (1,5,4)$, so $H$ has order 60.
$(1,4)(2,5)(1,2,3,4,5) = (3,5,4)$, so $H$ has order 60.
$(1,5)(2,3)(1,2,3,4,5) = (2,4,5)$, so $H$ has order 60.
$(1,5)(3,4)(1,2,3,4,5) = (2,3,5)$, so $H$ has order 60.
$(2,3)(4,5)(1,2,3,4,5) = (1,2,4)$, so $H$ has order 60.
$(2,4)(3,5)(1,2,3,4,5)^2 = (1,3,2)$, so $H$ has order 60.

Now, you'll notice that I've missed a few permutations, namely:

$(1,2)(3,5)$
$(1,3)(4,5)$
$(1,4)(2,3)$
$(1,5)(2,4)$
$(2,5)(3,4)$

With these, we have:
$(1,2)(3,5)(1,2,3,4,5) = (1,3)(4,5)$,
$(1,3)(4,5)(1,2,3,4,5) = (1,4)(2,3)$,
$(1,4)(2,3)(1,2,3,4,5) = (1,5)(2,4)$,
$(1,5)(2,4)(1,2,3,4,5) = (2,5)(3,4)$, and $(2,5)(3,4)(1,2,3,4,5) = (1,2)(3,5)$,

so if any one of these lies in $H$, all of them do. Further, these five elements, along with the powers of $(1,2,3,4,5)$ form a subgroup, of order 10. But if $H$ contains any other element, it must be one of order 2 or 5 (else $H$ would have order a multiple of 3): the order 2 cases have been covered above, and all give $H = A_5$. Checking the elements of order 5 that aren't already in there:

$(1,2,3,4,5)(1,2,3,5,4) = (1,3)(2,5)$,
$(1,2,3,4,5)(1,2,4,3,5) = (1,4)(2,5)$, $(1,2,3,4,5)(1,2,4,5,3)^2 = (1,5,4)$, $(1,2,3,4,5)(1,2,5,3,4)^2 = (1,3,2)$, $(1,2,3,4,5)(1,2,5,4,3) = (1,5,2)$, $(1,2,3,4,5)(1,3,2,4,5) = (1,4)(3,5)$, $(1,2,3,4,5)(1,3,2,5,4) = (1,5,3)$,
$(1,2,3,4,5)(1,3,4,2,5)^2 = (3,5,4)$,
$(1,2,3,4,5)(1,3,4,5,2) = (2,4)(3,5)$,
$(1,2,3,4,5)(1,3,5,4,2) = (2,5,3)$,
$(1,2,3,4,5)(1,4,2,3,5)^2 = (1,5,2)$,
$(1,2,3,4,5)(1,4,3,2,5) = (1,5,4)$,
$(1,2,3,4,5)(1,4,3,5,2) = (2,5,4)$,
$(1,2,3,4,5)(1,4,5,2,3)^2 = (2,4,3)$,
$(1,2,3,4,5)(1,4,5,3,2) = (3,5,4)$,
$(1,2,3,4,5)(1,5,2,3,4) = (1,3)(2,4)$,
$(1,2,3,4,5)(1,5,2,4,3) = (1,4,2)$,
$(1,2,3,4,5)(1,5,3,2,4) = (1,4,3)$,
$(1,2,3,4,5)(1,5,3,4,2) = (2,4,3)$,
$(1,2,3,4,5)(1,5,4,2,3) = (1,3,2)$.

So if $H$ contains any of these, it is all of $A_5$. Thus, there is no subgroup of $A_5$ of order 20, and hence no proper subgroup of order more than 12.

[NB: there are some pretty massive time-saving options available here: I went for keeping things as elementary as possible]