This is Proposition 21 on page 374 of Dummit and Foote's Algebra:
If $A,\ B$ are $R$-algebras, that is, they are module which have a center containing $R$, then $A\otimes_R B$ is still so where $R$ is commutative with $1$.
Proof : (1) $$r(a\otimes b)=(a\otimes b)r$$ That is, $R$ is in center of $A\otimes B$.
(2) Now we must prove that multiplication is independent of choice : $$ (ar\otimes br_2)(xr_3\otimes yr_4)=axrr_2r_3r_4 \otimes by \ \ast$$
(3) $1_A\otimes 1_B$ is identity in $A\otimes B$
(2') (2) can be replaced by the following : Define $f: A\times B\times A\times B\rightarrow A\otimes B,\ f((a,b,c,d))=ac\otimes bd $ Note that this map is multilinear.
Hence we have $A\otimes B\otimes A\otimes B = (A\otimes B)\otimes (A\otimes B )\rightarrow A\otimes B $ Hence $$ (A\otimes B)\times (A\otimes B )\rightarrow A\otimes B $$ is well-defined. So multiplication $\ast$ is well-defined.
So do I complete the proof ?
(2) is incomplete. Recall that not every element in the tensor product is a pure tensor.
In order to construct the algebra structure, I suggest the following (well-known) reformulation: If $A$ is some $R$-module, then an $R$-algebra structure on $A$ is the same as $R$-linear maps $\eta : R \to A$ and $\mu : A \otimes_R A \to A$ such that certain diagrams commute, asserting that $\eta$ is left- and right-unital for $\mu$ and that $\mu$ is associative. For example, associativity means that $$\begin{array}{c} A \otimes_R A \otimes_R A & \xrightarrow{\mu \otimes_R A} & A \otimes_R A \\ A \otimes_R \mu \downarrow ~~ && ~~\downarrow \mu \\ A \otimes_R A & \xrightarrow{\mu} & A \end{array}$$ commutes.
Now if $(A,\eta_A,\mu_A)$ and $(B,\eta_B,\mu_B)$ are two $R$-algebras, we obtain an $R$-algebra $(A \otimes_R B,\eta,\mu)$, where we define $\eta$ by $$R \cong R \otimes_R R \xrightarrow{\eta_A \otimes_R \eta_B} A \otimes_R B$$ and $\mu$ by $$(A \otimes_R B) \otimes_R (A \otimes_R B) \cong (A \otimes_R A) \otimes_R (B \otimes_R B) \xrightarrow{\mu_A \otimes \mu_B} A \otimes_R B.$$ You can check that the diagrams commute (using those for $A$ and $B$). Alternatively, you may calculate with elements in order to verify the algebra axioms. The whole point is that we have used general properties of the tensor product (e.g. associativity, symmetry) in order to construct the multiplication on $A \otimes_R B$. It is in fact given by $(a \otimes b) \cdot (a' \otimes b') = aa' \otimes bb'$, but we don't need to verify any well-definedness anymore. Also, the multiplication on $A \otimes_R B$ is $R$-bilinear by construction. We only have to check associativity of the multiplication and that $1 \otimes 1$ is a unit, which is verified in two lines.