This is an article which Alexandre Turull wrote. Lemma 2.1. states
Lemma 2.1. Suppose $H$ is a finite group, acting on the finite group $F$, and assume that $|H|$ and $|F|$ are relatively prime. Suppose that $K$ is an $H$-invariant subgroup of $F$. Then $|K : C_K(H)|$ divides $|F : C_F (H)|$.
and near the end of the proof it says
"(...) Hence, the minimality of our counterexample implies that $F$ is a $p$-group for some prime $p$. The minimality of |$F$ : $K$| in our counterexample implies that $K$ is maximal among the $H$-invariant subgroups of $F$. Let $\Phi$ be the Frattini subgroup of $F$. Since $K$ is a proper subgroup, $K\Phi$ is a proper $H$-invariant subgroup of $F$, and it follows that $\Phi \subseteq K$. Hence, $K$ is a normal subgroup of $F$."
I cannot see why the preceding deduction implies that $K$ is a normal subgroup of $F$.
First of all, I would like to say that the quoted part belongs to a larger proof, which does not appear here because I think that in the quoted part appears every hypothesis needed to proof the normality of $K$, but just in case I encorage you to read the whole proof (link below).
I have tried to get the result by proving that $K$ is a maximal subgroup of $F$, getting the normality due to the nilpotence of $F$, but this idea has not led me anywhere.
I also thought about using the H-invariant counterpart of the Sylow theorem's, but since $F$ is a $p$-group, it does not give me any information.
And finally, the property that if $F$ is a $p$-group, then $F/\Phi$ is elementary abelian, and since $\Phi \leq K$, $K/\Phi$ would also be, but once again, I doubt this helps solve the problem.
I would appreciate any help with this problem.
Thank you.
https://www.ams.org/journals/proc/2004-132-09/S0002-9939-04-07412-X/S0002-9939-04-07412-X.pdf
Since $F/\Phi$ is (elementary) abelian, all of its subgroups are normal. Therefore $K \Phi/ \Phi=K/ \Phi \lhd F/ \Phi$, which in turn implies that $K=K \Phi \lhd F$.