$S$ is a scheme, by an Abelian scheme over $S$ I mean a group scheme $X/S$ such that the structure morphism $X \rightarrow S$ is proper, flat and finite presentation. How to prove that $X$ is commutative?
My question is, given two $S$-morphism $f,g :Y \rightarrow Z$ between two $S$-schemes with $f_s=g_s, \forall s \in S$ (base change from $S$ to the residue field $k(s)$), under which assumption can we conclude $f=g$?
If this is not true in the proper and flat setting, how to prove that $X$ is commutative?
This is the first result of chapter 6 of Mumford's Geometric invariant theory, at least in the case that $S$ is locally Noetherian (and one can reduce to this case by standard limit arguments). It is indeed sufficient to ask for connected fibers (as Johann mentions), but in fact we can get away with less.
Let $f:Y \to Z$ be a morphism of (locally) finite type schemes over a connected locally Noetherian scheme $S$, with $Z$ is separated and $Y$ is proper and flat. Suppose that there is a point $s \in S$ such that $f(Y_s)$ is a point (set theoretically) and suppose that for all $s' \in S$ we have $h^0(X_{s'})=1$. Then there is a section $\eta:S \to Z$ such that $f$ factors over $\eta$. See here for a nice explanation of the proof.
One can apply this theorem to show that for an abelian scheme $A/S$, the map $(-1):A \to A$ is a group homomorphism, which implies that $A$ is a commutative group scheme. Let me point out that an abelian scheme is usually defined to be a proper smooth group scheme with connected (geometric) fibers, so your definition is definitely nonstandard.