Let $R \rightarrow S$ be a map of rings. This map is integral if and only if $R_{p} \rightarrow S_{p}$ is integral for every prime $p \subset R$. This implies that $R_p/ p R_p \rightarrow S_p / p S_p$ is an algebraic extension. Also, by Nakayama's lemma, $S_p / p S_p$ does not collapse to $0$.
Conversely, suppose $R_p / p R_p \rightarrow S_p / p S_p$ is an algebraic extension for each $p$, and that $S_p / p S_p$ is nonzero for each prime. Can we conclude that $R \rightarrow S$ is integral?