Multiple sources state that the cohomological Serre spectral sequence has a multiplicative structure $E^{p,q}_r \times E^{s,t}_r \to E^{p+s,q+t}_r$. For $r=1$, $E^{p,q}_1$ is $H^q(X_p,X_{p-1})$ and $E^{s,t}_1$ is $H^t(X_s,X_{s-1})$, but $E^{p+s,q+t}_1$ is $H^{q+t}(X_{p+s},X_{p+s-1})$. We can define the product structure $E^{p,q}_r \times E^{s,t}_r \to E^{p+s,q+t}_r$ by using cellular cohomological cross products (we can use cellular approximations to the spaces $X_p$ if need be) $H^q(X_p,X_{p-1}) \times H^t(X_s,X_{s-1}) \to H^{q+t}(X_p \times X_s,(X_p \times X_{s-1}) \cup (X_{p-1} \times X_s))$. We then use the fact that $H^k((X \times X)_m,(X \times X)_{m-1})=\oplus_{i+j=m}H^k(X_i \times X_j,(X_i \times X_{j-1}) \cup (X_{i-1} \times X_j))$, so $H^{q+t}(X_p \times X_s,(X_p \times X_{s-1}) \cup (X_{p-1} \times X_s))$ becomes one summand of $E^{p+s,q+t}_1$.
Every source I have encountered also states that this product structure is a derivation: that is, $d_r(xy)=d_r(x)y+(-1)^{p+q}xd_r(y)$. However, I have yet to see a proof of this. While the cellular cohomological cross product obeys $\delta(\phi \times \psi)=\delta(\phi) \times \psi+(-1)^k \phi \times \delta(\psi)$ for $\phi \in C^k(X;R)$ and $\psi \in C^l(Y;R)$, extending this results to the Serre spectral sequence seems nontrivial. I would greatly appreciate any assistance.