I recently found myself trying to prove (or disprove) the following lemma:
Lemma: Let H, K be groups and let $\varphi_1, \varphi_2 \colon K \rightarrow \mathrm{Aut}(H)$ be homomorphisms. Suppose also that $\varphi_1(K) \cong \varphi_2(K)$. Then $H \rtimes_{\varphi_{1}} K \cong H \rtimes_{\varphi_{2}}K$.
I believe I can prove this if there is an inner automorphism of $\mathrm{Aut}(H)$ that takes $\varphi_1(K)$ to $\varphi_2(K)$, however it feels like it should be true in general.
If anyone could shed some light on if/why this is true/false, or indeed provide me with any general criteria for determining whether two semi-direct products are isomorphic, I would be very appreciative.
Here's a counter-example with finite groups. Consider $H=S_4$, and $K=\Bbb Z/2$. Consider the morphisms $$\phi_i:\Bbb Z/2\to S_4\to\mathrm{Inn}(S_4)\subset\mathrm{Aut}(S_4)$$ where $\phi_1$ sends $1$ to the permutation $(12)$, and $\phi_2$ sends $1$ to the permutation $(12)(34)$, and the morphism $S_4\to\mathrm{Aut}(S_4)$ is of course the morphisms $g\mapsto\mathrm{conj}_g$, conjugation by $g$. The groups $X_1=S_4\rtimes_{\phi_1}\Bbb Z/2$ and $X_2=S_4\rtimes_{\phi_2}\Bbb Z/2$ aren't isomorphic, although the images of $\phi_1$ and $\phi_2$ are isomorphic as groups, both being isomorphic to $\Bbb Z/2$.
One way to see they cannot be isomorphic is to notice they don't have the same amount of elements of order $2$. I can give more details tomorow if you wish, but you can justify this to yourself, by figuring out what it means for an element to have order $2$ in $X_1$ and in $X_2$ respectively.
The abstract group structure of a subgroup of $\mathrm{Aut}(H)$ doesn't contain enough information about how the subgroup acts on $H$ for your conjecture to work, and I think it ought to be feasible to find examples with $\phi_1(K)=\phi_2(K)$ yet $X_1$ not isomorphic to $X_2$, even among finite groups.
EDIT : What I mean can be exemplified with the example above : the permutations $(12)$ and $(12)(34)$ both generate a subgroup of order $2$ of $\mathrm{Inn}(S_4)$, but although these groups are isomorphic, they are not equivalent in how they act on $S_4$ : they don't have the same number of fixed points for instance. This shows that the abstract group structure of a subgroup of an automorphism group (here the abstract group structure is $\Bbb Z/2\Bbb Z$) doesn't generally tell you everything there is to know about how its automorphisms act.
In other words, a single group may have several non-isomorphic faithful group actions on a given set.