Let $f:(X,\mathscr{O}_X)\to (Y,\mathscr{O}_Y)$ be a morphism of ringed spaces. Then for any $\mathscr{O}_X$-module $\mathscr{F}$ and $\mathscr{O}_Y$-module $\mathscr{G}$, there is a natural bijection of sets $$ \operatorname{Hom}_{\mathscr{O}_X}(f^*\mathscr{G},\mathscr{F})\cong \operatorname{Hom}_{\mathscr{O}_Y}(\mathscr{G},f_*\mathscr{F}) $$ giving the adjunction $f^*\dashv f_*$.
After some checking, I think more is true: First, we have $$ \operatorname{Hom}_{\mathscr{O}_X}(f^*\mathscr{G},\mathscr{F})\cong \operatorname{Hom}_{\mathscr{O}_Y}(\mathscr{G},f_*\mathscr{F}) $$ as $\Gamma(Y,\mathscr{O}_Y)$-modules, where $\operatorname{Hom}_{\mathscr{O}_X}(f^*\mathscr{G},\mathscr{F})$ has the $\Gamma(Y,\mathscr{O}_Y)$-module structure given by the contraction $\Gamma(Y,\mathscr{O}_Y)\to \Gamma(X,\mathscr{O}_X)$.
Therefore, it leads to me concluding that there is an isomorphism of $\mathscr{O}_Y$-modules $$ f_*\mathscr{H}om_{\mathscr{O}_X}(f^*\mathscr{G},\mathscr{F})\cong \mathscr{H}om_{\mathscr{O}_Y}(\mathscr{G},f_*\mathscr{F}) $$ and equivalently an isomorphism of $\mathscr{O}_X$-modules $$ \mathscr{H}om_{\mathscr{O}_X}(f^*\mathscr{G},\mathscr{F})\cong f^*\mathscr{H}om_{\mathscr{O}_Y}(\mathscr{G},f_*\mathscr{F}). $$
Please correct me if I am wrong, I want to be sure.
Your first isomorphism seems believable: just plug in any open set $U$ on both sides. You need to check things like $$(f^*\mathscr G)\big|_{f^{-1}U} = \left(f\big|_{f^{-1}U}\right)^* \left(\mathscr G\big|_U\right),$$ etc, because you want to use the adjunction $\left(f\big|_{f^{-1}U}\right)^* \dashv \left(f\big|_{f^{-1}U}\right)_*$.
But pulling it through the adjunction only gives you a morphism$$f^*\mathscr{H}om_{\mathcal O_Y}(\mathscr G, f_*\mathscr F) \to \mathscr{H}om_{\mathcal O_X}(f^*\mathscr G, \mathscr F).$$ You cannot expect this map to be an isomorphism (unless the counit for the adjunction is an isomorphism, which is rarely the case).
To give an example where the second map is not an isomorphism, consider $\mathscr G = \mathcal O_Y$. Then the last morphism is the counit $f^*f_* \mathscr F \to \mathscr F$.
For example, consider $X = \mathbb P^1_k$, and $Y = \operatorname{Spec} k$, and let $\mathscr F = \mathcal O(-1)$. Then $$f_* \mathscr F = H^0(\mathbb P^1, \mathscr F) = 0,$$ so $f^*f_* \mathscr F = 0$. Thus, $f^*f_* \mathscr F \to \mathscr F$ is not an isomorphism, as $\mathscr F \neq 0$.