Can I recover a group by its homomorphisms?

Question

There is finitely generated group $G$ which I don't know. For every finite group $H$ I can think of, I know the number of homomorphisms $G \to H$ up to conjugation. (By this I mean that two homomorphisms $\phi_1$ and $\phi_2$ are being considered equivalent if there is a $h \in H$ such that $\phi_1(g)h = h\phi_2(g)$ for all $g \in G$.)

Given these numbers, do I have enough information to recover $G$?

Edit: The question is motivated from physics. A flat $H$-connection on a manifold $M$ is a homomorphism $\pi_1(M) \to H$ and a gauge transformation is a conjugation. So I'm interested whether I can recover the fundamental group by counting equivalence classes of connections, for arbitrary finite gauge groups.

Edit 2: It would be interesting as well if we count the number of homomorphisms without taking the equivalence by conjugation into account.

Answer 1

No. If $G$ is infinite and simple, then any homomorphism $f:G \to H$ for $H$ finite must be trivial; otherwise, $\ker f$ would be a proper normal subgroup of $G$. Such (finitely-generated) groups exist, though their construction is nontrivial; look at the Thompson groups, for example, or one of Higman's examples.

Answer 2

For $G$ finite the answer is yes.

It is clearly enough to know which finite groups $H$ are quotients of $G$, as $G$ will the biggest of them. For this we determine the number of surjective homomorphisms $G \to H$ by induction: For $H=1$ there is exactly one. So we may assume that we know the number of surjective homomorphisms $G \to U$ for all proper subgroups $U$ of $H$.

By assumption we are given the number $n_H$ of homomorphisms $\varphi:G \to H$ up to conjugation by elements $h$ of $H$ where $\varphi^h(g) = (\varphi(g))^h$. Let $U = \varphi(G)$ be the image of $\varphi$ in $H$. For $h\in C_H(U)$ centralizing $U\le H$ we have $\varphi^h = \varphi$, and vice versa the equality implies that $h$ centralizes $U$. So counting modulo conjugation is the same as weighing each homomorphism with one over the index of the centralizer of its image. (For $h\in N_H(U)$ normalizing $U\le H$ we know $\varphi^h(G)=\varphi(G)$. For general $h\in H$ we get that that the image of $G$ in $H$ under $\varphi^h$ is a subgroup of $H$ conjugated to $U$: $\varphi^h(G) = U^h$.)

Writing the number $n_H$ of homomorphisms (up to conjugacy) as $$n_H = \sum_{\varphi:G\to H} \frac{|C_H(\mathrm{Im}(\varphi))|}{|H|} = \sum_{U\le H}\sum_{{\varphi:G\to U} \mbox{ surjective}} \frac{|C_H(U)|}{|H|}$$ the left side $n_H$ is known by assumption. By induction all but one term (for $U=H$) of the outer sum on the right side can be calculated, which enables us to determine the missing term $x = \sum_{{\varphi:G\to H} \mbox{ surjective}} \frac{|C_H(H)|}{|H|}$. The number of surjective homomorphisms $G\to H$ is then $x$ times the index of the center of $H$.

The proof can easily be adapted for the case without conjugation mentioned in edit 2.