Given that $\phi: G\to H$ is a homomorphism, and that $K \leq H$, and also that $\phi^{-1}(K) := \{g \in G: \phi(g)\in K\}$; if one wants to prove that $\phi^{-1}(K)$ is closed under product and taking inverses, would it be sufficient to say the following?
For some $g_1 g_2 \in G$, such that $\phi(g_1) \in K$ and $\phi(g_2) \in K$, we have that $\phi(g_1 g_2^{-1}) = \phi(g_1) \phi(g_2)^{-1}$. And $\phi(g_1)\phi(g_2)^{-1}$ must be in $K$ since $K \leq H$ and $H$, being a group, is closed under product and taking inverses.
Therefore, $g_1$ and $g_2^{-1}$ are in the pullback of $K$, since for $\phi(g_1)\phi(g_2)^{-1} = k \in K$, $\phi^{-1}(k) \in G$.
Please let me know if you think this argument is sound enough.
You have the right idea and can make the argument a bit clearer by stating the subgroup criterion you're using. Indeed, what you're relying on implicitly is that, in general, if $G$ is a group and $K_0$ a subset of $G$ then $K_0$ is a subgroup of $G$ if and only if $k_1k_2^{-1} \in K_0$ whenever $k_1,k_2 \in K_0$. I've provided an explicit argument hidden by a spoiler tag below.