Let $R$ be a commutative ring and $G$: Alg$_{/R}\to $Ab an abelian fppf sheaf. Let SmoothAlg$_{/R}$ be the category of smooth $R$-algebras. Is the value of $G$ determined by the value of the restriction of $G$ to SmoothAlg$_{/R}$?
It would be enough to show that any $R$-algebra has an fppf covering by smooth $R$-algebras, but I can’t seem to prove/disprove this result or find a good reference for it.
I would also appreciate any general reference about the fppf topology / fppf sheaves. I’m happy to read things in EGA/SGA.
This is definitely not true.
While one can say something more generally, I think it's instructive to restrict ourselves to the assumption that:
$R=k$, a field which, for simplicity, I will assume is perfect,
$G$ is representable by a finite-type group $k$-scheme (also denoted $G$) -- note that the fppf topology is subcanonical (i.e., every representable presheaf is a sheaf).
If $G$ is a smooth group $k$-scheme, then the answer to your question is evidently yes by a Yoneda argument. If $G'$ were another smooth group $k$-scheme such that $G$ and $G'$ take the same values on $\mathrm{SmoothAlg}_k$, then there is an isomorphism of $G$ and $G'$ in the category of presheaves on $(\mathrm{SmoothAlg}_k)^\mathrm{op}$ which, as $G$ and $G'$ are actually still representable in this smaller subcategory of $(\mathrm{Alg}_k)^\mathrm{op}$, they must be isomorphic by Yoneda.
In fact, I claim that this recoverability of $G$ from its values on $\mathrm{AlgSmooth}_k$ is essentially equivalent to being smooth in some sense. Namely, if $G$ is now just any representable sheaf, then observe that any smooth $k$-algebra $S$ is reduced, and so
$$G(S)=\mathrm{Hom}(\mathrm{Spec}(S),G)=\mathrm{Hom}(\mathrm{Spec}(S),G_\mathrm{red})=G_\mathrm{red}(S),$$
where $G_\mathrm{red}\subseteq G$ is the reduced subscheme (which is still a group scheme -- see [Milne, Corollary 1.39]). But, observe that $G_\mathrm{red}$ is actually smooth (see [Milne, Proposition 1.26 and Proposition 1.28]).
So, combining the above observations, we can conclude the following.
In characteristic $0$ every such $G$ is reduced (e.g., see [Milne, Theorem 3.23]), but there are many non-reduced groups if the characteristic of $k$ is $p>0$ (e.g., $\mu_p$).
You were correct though, that this claim would follow if every (presumably you wanted finite type) $k$-algebra had an fppf cover by smooth $k$-algebras. But, this can never happen unless the algebra is itself smooth -- the fppf topology is quite good at remembering structural properties. For instance, Tag 033D is all about properties that descend along an fppf cover and they include, for instance, reduced, normal, and regular.
For places to read about the fppf topology, I am not sure what you are trying to learn about. Are you trying to learn about the formalism of sheaves on a site, or about what properties hold for the fppf site? If it's the former then I think any good book on the topic will do (e.g., I like Milne's published book on etale cohomology -- not his online notes). If it's the latter, I don't know if there is a really a good reference, as it's a bit of technical topic. In fact, the Stacks Project might be one of your best bets.
References:
[Milne] Milne, J.S., 2017. Algebraic groups: the theory of group schemes of finite type over a field (Vol. 170). Cambridge University Press.