$\require{AMScd}$ I understand that given a group $G$, all homomorphisms from $G$ to an abelian group factor through the abelianization of $G$, i.e. maps $G\to A$ factor through projection $G\to G/\left[G, G\right]$.
I was wondering if you could define the same for codomain groups that are not abelian. I am not even sure if “largest non-abelian factor” is well-defined, but I came up with this idea.
Consider $G_0 = G$ in the sequence $\require{tikzcd}$ \begin{CD} G_0 @>\zeta_1>> G_1 @>\zeta_2>> \cdots \end{CD} where $G_{i + 1}$ is $G_i / Z(G_i)$ and $\zeta_i$ is projection mod $Z(G_i)$ (alternatively, visualize a herringbone of ses's).
What if at some point \begin{CD} \cdots @>\zeta^*>> G^* @>\zeta{^*}'\, \overset{\huge\color{red}?}{=} \,\,\mathrm{id}\,>> G^* \end{CD}
Certainly this sequence must end if $G$ is finitely generated. If that isn't the case, then my question concerns $G^* = \displaystyle\lim_{\longrightarrow} G_i$.
What exactly lies at the end of this tunnel? As far as I can tell, projection onto $G^*$ uniquely factors all epimorphisms $G\to \text{a non-abelian group}$. Is that right?
Is $Z(G)$ dual to $[G, G]$? What does $G^*$, if it exists, say about $G$?
Already your favourite non-abelian group of order $p^3$ ($p$ any prime) and the identity function show that taking the epimorphisms to non-abelian groups does not work (as commented by Grumpy Parsnip).
Try epimorphisms to center-free groups instead!
For understanding your $G^*$ take a look at the kernels of your maps $\zeta_1:G_0\to G_1$, $\zeta_2\circ\zeta_1:G_0\to G_2$, $\zeta_3\circ\zeta_2\circ\zeta_1:G_0\to G_3$, $\dots$, which are the elements of the upper central series (read also the following section "Connection between lower and upper central series" in the wikipedia for your last question).