Defining a subgroup of $\mathrm{Aut}(G)$ in GAP

Question

Let $G$ be a finite group and let $A$ be a normal subgroup of $G$. What I would like to do is construct the following subgroup of $\mathrm{Aut}(G)$ in GAP:

$$ X(G,A) = \{ \phi \in \mathrm{Aut}(G): g^{-1}\phi(g) \in A\quad \forall g \in G\}.$$

Now there should be a straightforward way to do this: iterate over all elements of $\mathrm{Aut}(G)$ and check which of them satisfy $g^{-1}\phi(g) \in A$ for all $g \in G$. Append each such element to a list and in the end, construct the subgroup that the elements in the list generate.

Although I have not yet implemented this, I am wondering if there might be a "slicker" way to construct the subgroup I want. It is, of course, unclear what exactly "slicker" means here, but I feel an expert in GAP would probably do this in some other, more elegant fashion.

Answer 1

The core idea is to only ever test one element in every coset of the subgroup you have found so far. Either it satisfies the condition (in which case you increase the subgroup, and the whole coset is in) or it does not (in which case no other element in the same coset can do so. You get for free a one-sidedly irredundant generating set (i.e. no generator is in the span of the previous ones).

Such functionality is available in GAP for permutation groups as SubgroupProperty, and one can use an isomorphism to a permutation group (Note that many operations for groups of automorphisms anyhow do so) to make use of it, viz.:

MyX:=function(G,A)
local aut,iso,prop,s;
  aut:=AutomorphismGroup(G);
  iso:=IsomorphismPermGroup(aut);
  prop:=function(phi)
    phi:=PreImagesRepresentative(iso,phi);
    return ForAll(GeneratorsOfGroup(G),
      g->LeftQuotient(g,ImagesRepresentative(phi,g)) in A);
  end;
  s:=SubgroupProperty(Image(iso),prop);
  return PreImage(iso,s);
end;

Then (for example):

gap> g:=SymmetricGroup(4);;
gap> a:=Socle(g);
Group([ (1,4)(2,3), (1,2)(3,4) ])
gap> MyX(g,a);
<group with 2 generators>
gap> Size(last);
4