Define for each morphism $\varphi: G \to H$ a corresponding morphism $\varphi^{\text{ab}}: G^{\text{ab}}\to H^{\text{ab}}$ with the properties:
- $(\text{id}_{G})^{\text{ab}} = \text{id}_{G^{\text{ab}}}$
- $(\psi \circ \varphi)^{\text{ab}} = \psi^{\text{ab}} \circ \varphi^{\text{ab}}$.
Here $G^{\text{ab}} = G/G' = G/[G,G]$.
I wonder if the question is even well defined, since it does not define $\psi$. I guess $\psi$ should be a morphism from $H$ to another group $L$, or $\psi: H \to L$?
My first idea was to define $\varphi^{\text{ab}}$ in a natural way, check the requirements and then check if it's well defined. Like so: $$ \begin{align} \varphi^{\text{ab}}= G/G' & \to H/H'\\ gG'&\mapsto g^\varphi {G'}^\varphi \end{align} $$
But this has issues, since ${G'}^\varphi$ is not necessarly the commutator subgroup of $H$. I know that ${G'}^\varphi \leq H'$
Second idea was to define $\varphi^{\text{ab}}$ $$ \begin{align} \varphi^{\text{ab}}= G/G' & \to H/H'\\ gG'&\mapsto g^\varphi H' \end{align} $$
But this does not meet the first requirement. $\text{id}_{G^{\text{ab}}}: G/G' \to G/G'; gG' \mapsto gG'$, while here $(\text{id}_G)^{\text{ab}}$ would map $gG'$ to $gH'$.
I wonder if this question is well defined, and if so, what other ways would there exist to mitigate the issue of ${G'}^{\varphi} \leq H'$.
Your second idea is actually correct : in the case of the identity, you would have $H=G$ and thus $H'=G'$.
Another way to do this would be to use the universal property of $G^{ab}$: for every group homomorphism $f:G\to A$ with $A$ abelian, there exists a unique homomorphism $\tilde{f}:G^{ab}\to A$ such that $f=\tilde{f}\circ \pi_{G'}$, where $\pi_{G'}$ is the quotient map of the subgroup $G'$. In particular, you can take $A=H^{ab}$ and $f=\pi_{H'}\circ \phi$, and then you obtain a unique group homomorphism $\phi^{ab}:G'\to H'$ such that $\phi^{ab}\circ \pi_{G'}=\pi_{H'}\circ \phi$. This is equivalent to what you wrote, and using the uniqueness requirement, you can then prove that this has the required properties. For example, given another homomorphism $\psi:H\to L$, we have $$ \psi^{ab}\circ\phi^{ab}\circ\pi_{G'}=\psi^{ab}\circ\pi_{H'}\circ \phi=\pi_{L'}\circ \psi\circ \phi,$$ hence $\psi^{ab}\circ\phi^{ab}=(\psi\circ \phi)^{ab}$. The case of the identity morph ism is similar.