Assume $R$ is a unital ring, and that $M$ has a left-module structure. Furthermore, we are given an action $$(r\varphi)(r') = \varphi(r'r)$$ on $$\operatorname{Hom}_{\mathbb{Z}}(R,M).$$
Suppose that $$0 \to A \to B$$ is an exact sequence, such that for all homomorphisms $$f \in \operatorname{Hom}_{R}(A,M)$$ there is a lift $$F \in \operatorname{Hom}_{R}(B,M)$$ such that $$f = F \circ \psi.$$ Then for every homomorphism $$f' \in \operatorname{Hom}_{R}(A,\operatorname{Hom}_{\mathbb{Z}}(R,M))$$ there exists a lift $$F' \in \operatorname{Hom}_{R}(B,\operatorname{Hom}_{\mathbb{Z}}(R,M))$$ such that $$f' = F' \circ \psi$$
Now, by the hint, I should look at $$f(a) = f'(a)(1_{R})$$ and show that then $$F'(b)(r) = F(rb)$$ defines a lift.
Now, clearly $$f'(a)(1_R) \in M$$ and $$F'(b)(1_R) = F(b) \in M$$ so that $$(F' \circ \psi(a))(1_R) = F'(\psi(a))(1_R) = F'(b')(1_R) = F(b') = f(a)$$ where $\psi(a) = b'$.
Furthermore, we see that $$(rf'(a))(1_R) = rf(a) = f(ra) = f'(ra)(1_R)$$ since by assumption $f$ is an $R$-module homomorphism.
And $$F'(r'b)(1_R) = r'F(b) = r'F(b)(1_R)$$ since by assumption $B$ is an $R$-module and $F$ is an $R$-module homomorphism.
Now, to conclude, I believe you can say that a $\mathbb{Z}$-module homomorphism is completely determined by where it ends $1_R$, so the fact that $(F' \circ \psi)(a) \in \operatorname{Hom}_{\mathbb{Z}}(R,M)$ and $f'(a) \in \operatorname{Hom}_{\mathbb{Z}}(R,M)$ agree on $1_R$ shows that they will agree on every other element in their respective domain.
Am I missing something in this explanation? I am unsure. I am not using the injectivity of $\psi$ here.