Question: Prove that if $G\leq S_n$ of index $2$, then $G=A_n$.
Now, I don't want to appeal, if I can avoid it, to the fact that $A_n$ is simple for $n\geq 5$. Moreover, I want to see if I can use the method below. I know that it can be done by induction on $n$ while assuming $G\neq A_n$ and $|S_n:G|=2$, so there is at least one $3$ cycle not in $G$, then assuming $(123)$ is not in $G$, we get $3$ distinct cosets, which is a contradiction.
However, I want to try and prove it in the following way, because I feel like it is a more "common" technique.....
Suppose $G\neq A_n$, then since $A_n$ is normal in $S_n$ and $G\leq S_n$, we have that $GA_n=S_n$, thus $\frac{|G||A_n|}{|G\cap A_n|}=|S_n|$. Now, I can either divide both sides by $|G|$ or by $|A_n|$, and I get whatever is on the LHS is equal to $2$. However, I don't see how to generate a contradiction whether I consider $|G:G\cap A_n|$ or $|A_n:G\cap A_n|$. I think I am missing something pretty trivial, but I just can't see it. Any help is greatly appreciated! Thank you.
Too long for a comment. I don't know if you will accept this as an answer, since it does not follow your method. However, as Arturo says, if you don't use anything about the structure of $A_n$ or $S_n$, you won't be able to onclude.
Here is an argument which does not need anything fancy about $A_n$ ( commutators, or simplicity...)
The only thing you need to know is that there is a unique non trivial group morphism $S_n\to \{\pm 1\}$, namely the signature morphism.
Let me recall the argument: transpositions are all conjugate in $S_n$ ( $\tau (ij)\tau^{-1} =(kl)$, where $\tau\in S_n$ satisfies $\tau(i)=k, \tau(j)=l$), so transpositions under such a morphism all have same image (since $\{\pm \}$ is abelian. Since transpositions generate $S_n$, this ismage is $-1$, and the morphism is the signature.
Once you have that in mind, this is quite easy. Let $H$ be a subgroup of index $2$ of $S_n$; Then $H$ is normal in $S_n$ (classical result, valid for any index $2$ sungroups of a given group $G$), and we have a group isomorphism $S_n/H\simeq \{\pm 1\}$. This isomorphism sends elements of $H$ to $1$, and the other ones to $-1$.
Then the composition $S_n\to S_n/H\simeq\{\pm 1\}$ is a non trivial morphism, with kernel $H$ by definition. But this non trivial morphism is the signature morphism, so its kernel is $A_n$. Consequently, $H=A_n$.
Side remark. More general, there is a $1-1$ correspondence between subgroups of index $2$ of a given group $G$ and the set of non trivial morphisms $ G\to \{\pm 1\}$.