Given a topological space $X$ and two sheaves $F,G$ on it, where both sheaves take value in some category $\mathcal{C}$. My initial task is to prove in the case both $F,G$ take value in $\mathsf{Set}$, that $\text{Hom}(F,G)$ is a sheaf on $X$ with value in $\mathsf{Set}$. I can prove this, but I want to know more about the case where $\mathcal{C}$ is not $\mathsf{Set}$.
At the step to check the glue property of sheaves, I would have to construct a $\phi(V)\in \text{Hom}(F,G)(V):=\text{Hom}(F|_V,G|_V)$ by gluing up compatible morphisms $\phi(U_i) \in \text{Hom}(F,G)(U_i)$, with $\{U_i\}_{i\in I}$ an open cover of $V$.
One obvious choice is to define it value-wisely, $$ \phi(V)(f):=\text{ the unique section $g$ in $G(V)$ that }g_i:=\phi(U_i)(f|_{U_i}) \text{ glue up to}. $$ The $\phi(V)$ defined in this way is indeed a morphism in the category $\mathsf{Set}$ and seems to be also true for many familiar categories of structured sets. My question: Is this value-wise defined $\phi(V):f\mapsto g$ always a well-defined morphism in general category $\mathcal{C}$? What property do we require the category $\mathcal{C}$ to have to make sure the gluability of morphisms?
Well, what you have written definitely doesn't make sense in a general category $\mathcal{C}$. After all, objects of $\mathcal{C}$ may not be sets, and morphisms of $\mathcal{C}$ may not be functions, so you can't define a morphism $\phi(V)$ by just defining $\phi(V)(f)$ for each "element" $f\in F(V)$.
However, you can turn it into something that makes sense using the Yoneda embedding. We identify $\mathcal{C}$ as a full subcategory of the functor category $\mathsf{Set}^{\mathcal{C}^{op}}$, thinking of an object of $\mathcal{C}$ as having "elements" which are just all the possible maps from other objects. The key fact then is that the Yoneda embedding preserves limits and limits in the functor category $\mathsf{Set}^{\mathcal{C}^{op}}$ are computed pointwise. This means that the sheaf condition for a functor $F$ taking values in $\mathcal{C}$ is equivalent to just the ordinary sheaf condition for sets on the Hom-sets $\operatorname{Hom}_{\mathcal{C}}(T,F(-))$ for each object $T$ of $\mathcal{C}$.
So, your proposed definition of $\phi(V)$ actually ends up working for any category $\mathcal{C}$, provided you interpret the "element" $f$ of $F(V)$ as a morphism $T\to F(V)$ where $T$ is some object of $\mathcal{C}$. There are of course some details to check that this actually works (for instance, you have to define $\phi(V)$ not just on $F(V)$ but on $F(U)$ for every open subset $U\subseteq V$ and check that these are compatible to define a morphism of sheaves $F|_V\to G|_V$), but everything does work out and you can conclude that $\operatorname{Hom}(F,G)$ is a sheaf.
Alternatively, you can just use the limit condition in the definition of a sheaf directly. Since $G(V)$ is the limit of the diagram formed by the objects $G(U_i)$ and their intersections, to define a morphism $F(V)\to G(V)$ you just have to give compatible morphisms $F(V)\to G(U_i)$. These morphisms are just obtained by composing the restrictions $F(V)\to F(U_i)$ with the given morphisms $F(U_i)\to G(U_i)$. Of course, there are again a lot of details to be check to verify this really does make $\operatorname{Hom}(F,G)$ a sheaf.