A subpresheaf $Y$ of $X$ is a presheaf $Y$ such that for all $C$ $YC \subseteq XC$ and $Yf$ is $Xf$ restricted to $Y(cod(f))$. In my lecture notes is the following quote:
"What do subpresheaves of $\hom(-, C)$ look like? If $R$ is a subpresheaf of $\hom(-, C)$, then $R$ can be seen as a set of arrows with codomain $C$ such that if $f: C'\to C$ is in $R$, and $g: C'' \to C'$ is arbritrary, then $fg$ is in $R$ (for $fg = \hom(f, C)(g)$). "
I don't understand the last part.
As you said, a subpresheaf of $\operatorname{hom}(-, C)$ is a presheaf $Y$ such that for each $C'$ we have $YC' \subseteq \operatorname{hom}(C', C)$. So $Y$ is just a collection of arrows with codomain $C$. But it is not just any such collection, because for $g: C'' \to C'$ we must have that $Yg$ is the restriction of $\operatorname{hom}(g, C)$. The latter is just given by precomposition: it takes some $f: C' \to C$ in $\operatorname{hom}(C', C)$ and outputs $fg: C'' \to C$ in $\operatorname{hom}(C'', C)$. So it should be $Yg(f) = fg = \operatorname{hom}(g, C)(f)$ and not what you wrote in your question. But then $fg \in YC''$, so since $g$ was arbitrary we have that $Y$ is closed under precomposition.
So a subpresheaf $Y$ of $\operatorname{hom}(-, C)$ gives us a collection of arrows into $C$ that is closed under precomposition. Conversely, given a collection $R$ of arrows into $C$ we can define a subpresheaf $Y$ of $\operatorname{hom}(-, C)$ by setting $YC' = \{f \in R : \operatorname{dom}(f) = C'\}$ and then $Yg$ is of course the restriction of $\operatorname{hom}(g, C)$ for any $g: C'' \to C'$. The fact that this is well-defined is due to the fact that $R$ was assumed to be closed under precomposition.