Irreducible $G$-group, Irreducible $G$-module.

Question

Let $G$ and $M$ be two groups. Let $M$ be a $G$-group. What does an irreducible $G$-group mean? I have heard the term irreducibility for modules. So, what actually irreducibility means here? Is that something related to $G$-module? However, when we say $M$ is a $G$-module, in that case, $M$ has to be an abelian group. Here, $M$ is an arbitrary group. It does not make sense to me. Please help me with this. For the definition of $G$-groups, you can refer to https://en.wikipedia.org/wiki/Group_with_operators. For the definition of $G$-modules, you can refer to https://en.wikipedia.org/wiki/G-module.

Answer 1

In the paper

Lafuente, Julio P.; Lizasoain, I.; Ochoa, Gustavo, Projective G-groups, J. Pure Appl. Algebra 165, No. 2, 213-225 (2001). ZBL1017.20019,

an irreducible $G$-group is defined to be a group with an action of $G$ by group automorphisms that has no proper nontrivial $G$-invariant normal subgroups.

That doesn't prove that this is the same as the definition intended in the paper

Detomi, E.; Lucchini, A., Crowns and factorization of the probabilistic zeta function of a finite group., J. Algebra 265, No. 2, 651-668 (2003). ZBL1072.20031,

where the OP said (in comments) that he encountered the term, but that paper refers to another paper co-authored by Lafuente for other terminology regarding $G$-groups, so it seems very likely that this is the intended definition.

Answer 2

I've never seen 'an irreducible $G$-group' for an arbitrary group $M$. However, there is at least a notion of a simple $G$-group for a (group) representation $\pi \colon G \to \operatorname{Aut}M$; see [Aschbacher 2000, pp. 22f.], for instance. I think this is what it means as D. Holt already commented. (Added. I mean, a group $M$ without non-trivial proper $G$-invariant normal subgroup as J. Rickard noted.)

If $M$ is abelian, then one can extend the action of $G$ on $M$ and regard $M$ as a $\mathbb ZG$-module (and vice versa). So you can (comfortably?) talk about simplicity/irreducibility. Or you may want to see everything from the viewpoint of abstract representations discussed in pp. 9f. in the above book.