Let $X, Y, Z$ three $k$-varieties. By "variety over field $k$" I mean a separated $k$-scheme of finite type which is geometrically integral.
Suppose that $f: X \to Z, g: X \to Y$ and $h: Y \to Z$ are morphisms and there is an open dense subvariety $U \subset X$ such that the restriction of $f$ to $U$ factorizes as $f \circ rest(U)= h \circ g \circ rest(U)$ where $rest(U): U \to X$ is the canonical open embedding.
Why this implies that then the morphism $f$
factorizes as $f= h \circ g$? Note that
$Y$ is not assumed to be proper therefore
the valuative criterion to force
$f$ factorize at every point $x \in X$ over $Y$
not works.
In the original text (Abelian varities, Lemma
1.11 ("Rigidity") uses that $X$ is irreducible.
Why this serves the implication?
Consider $i:W \to X$, as the base change of $\Delta: Z\to Z\times Z$ by $(f ,h\circ g): X\to Z\times Z$.
Now $\Delta$ is a closed immersion (since $Z$ is seperated over $\operatorname{Spec} k$), implies $i:W\to X$ is a closed immersion.
Further, consider the scheme theoretic image $i(W)$ of $i$ in $X$, it is a closed subset of $X$ with scheme structure from that of $W$. On the other hand, since $X$ is reduced (hence $U$ is reduced), the scheme theoretic image $Im(j)$ of $j: U\to X$ is $X$ itself. (see Exercise 3.11(d), Chapter 2, Hartshorne)
Since $j:U\to X$ factorizes through $W\to X$, the image schemes $i(W)$ and $X$ have the same underlying topological spaces and are isomorphic as schemes. (See Exercise 3.11(c), Chapter 2, Hartshorne and use sort of `universal property' of scheme theoretic image)
Thus, $f=h\circ g$ on $X$.
This is essentially the Exercise 4.2 Chapter II, Hartshorne.