Let $H$ be any group and $K$ an abelian group. (I'm interested in $K={\mathbb Z}$.) Homomorphisms $H\to Aut(K)$ define semi-direct products $K\rtimes H$.
There is an action of $Aut(H)\times Aut(K)$ on $Hom(H,Aut(K))$, where $Aut(K)$ acts on itself by conjugation. Homomorphisms in the same $Aut(H)\times Aut(K)$-orbit yield isomorphic semi-direct products. (Isomorphic as a group, of course not as an extension.)
I'm interested in a converse. In the theory of finite groups it is known that under the assumption $gcd(\mid H\mid,\mid K\mid)=1$ two semi-direct products are isomorphic only if the corresponding homomorphisms $H\to Aut(K)$ belong to the same $Aut(H)\times Aut(K)$-orbit. (Unfortunately I don't have a reference for this fact.)
My question is for $K={\mathbb Z}$ and infinite groups $H$. (Say free groups or surface groups.) Can there be isomorphic semi-direct products in different $Aut(H)\times Aut(K)$-orbits?