The following assertion is true:
If $A$ and $B$ are dense group homeomorphic subgroups of complete Hausdorff topological groups $X$ and $Y$ respectively, then $X$ and $Y$ are group homeomorphic.
I wonder if we can find $A$ and $B$ such that $A$ is homeomorphic to $B$, but not group homomorphic and that $A$ and $B$ are dense in topological groups $X$ and $Y$ respectively, but $X$ and $Y$ are not homeomorphic.
Then what about $X$ and $Y$ are homeomorphic but not group homomorphic?
The the topological groups are assumed to be commutative, by group homeomorphic we mean that (topological) homeomorphic that is also (group) homomorphic.
Sure, there's lots of examples.
$A$, $B$ homeomorphic, not group homeomorphic, and $X, Y$ not homeomorphic: Take $X=\mathbb{R}$, $Y=\mathbb{R}^2$ with the usual topologies and group structures. Let $A=\mathbb{Q}\subseteq X$ and $B=\mathbb{Q}^2\subseteq Y$. Then $A$ and $B$ are dense in $X$ and $Y$ respectively, are topological subgroups, and are homeomorphic (this might be a bit surprising, but it's true, and a good exercise) - but clearly $X$ and $Y$ are not homeomorphic!
$A$, $B$ homeomorphic, $X$, $Y$ homeomorphic, but $X$, $Y$ not group homeomorphic: Consider the sets $2^\omega$ and $3^\omega$ of infinite strings from $\{0, 1\}$ and infinite strings from $\{0, 1, 2\}$ respectively. These sets carry natural group structures - the $\omega$-fold products of $\mathbb{Z}/2\mathbb{Z}$ and $\mathbb{Z}/3\mathbb{Z}$, respectively. They also have a natural topological structure (the product topology, with the discrete topology on each fact), and form topological groups. As spaces, they are homeomorphic, but obviously they are not group isomorphic; these will be our $A$ and $B$. Now let $X$ and $Y$ be arbitrary countable dense subgroups of $A$ and $B$, respectively. We will have $X$ and $Y$ homeomorphic as topological spaces.