I know that the algebraic tensor product of $\mathbb{K}$-Hilbert spaces $A$ and $B$ is defined as $F(A \times B)/R$, where $R$ is the subspace generated by elements of the form $\lambda_{1}(a_{1},b) + \lambda_{2}(a_{2},b) - (\lambda_{1}a_{1}+\lambda_{2}a_{2}, b)$ and $\lambda_{1}(a,b_{1}) + \lambda_{2}(a, b_{2}) - (a,\lambda_{1}b_{1} + \lambda_{2}b_{2})$. In other words, if $a \otimes b = (a,b)+R$, then we have: $$ (a_{1} + a_{2}) \otimes b = a_{1} \otimes b + a_{2} \otimes b $$ $$ a \otimes (b_{1}+b_{2}) = a \otimes b_{1} + a \otimes b_{2} $$ $$ \lambda( a \otimes b) = (\lambda a) \otimes b = a \otimes (\lambda b) $$
Now, even though $A$ and $B$ are Hilbert spaces, $A \otimes_{alg} B$ is still just an ordinary vector space with this definition, but with an inner product defined as $\langle a_{1} \otimes b_{1}, a_{2} \otimes b_{2} \rangle = \langle a_{1}, a_{2} \rangle \langle b_{1}, b_{2} \rangle$, and expanded into sesquilinearity, we get a prehilbert space. The completion of this space is the Hilbert space $A \otimes B$.
Still, with this definition, I still only know that finite linear combinations can go through $\otimes$, i.e. that $$ (\lambda_{1}a_{1}+...+\lambda_{n}a_{n}) \otimes b = \lambda_{1}(a_{1} \otimes b) + ... + \lambda_{n}(a_{n} \otimes b). $$
What about infinite sums? Is there a step in the aforementioned construction that lets me apply distributivity with regard to infinite sums, i.e. some kind of continuity of the $\otimes$ sign?
Suppose I wanted to prove that if $\{ a_{i} \}_{i \in I}$ is a Hilbert base for $A$ and that $\{ b_{j} \}_{j \in J}$ is a Hilbert base for $B$, then $\{ a_{i} \otimes b_{j} \}_{i \in I, j \in J}$ is a Hilbert base for $A \otimes B$:
Suppose I want to prove that these elements span $A \otimes B$: $ X = \sum_{k=1}^{n} \lambda_{k}a_{k}' \otimes b_{k}'$ is an arbitrary element of $A \otimes B$, and $a_{k} = \sum_{i \in I} \langle a_{k}', a_{i}\rangle a_{i}$, and $b_{k} = \sum_{j \in J} \langle b_{k}', b_{j}\rangle b_{j}$ (only at most countably many of these are non-zero in both sums, a standard result). Then I'd have $$ X = \sum_{k=1}^{n} \lambda_{k}(\sum_{i \in I}\langle a_{k}', a_{i}\rangle a_{i}) \otimes (\sum_{j \in J} \langle b_{k}', b_{j}\rangle b_{j}). $$ But I don't know if I can use distributivity in this situation. Is this a case of having to define $R$ to be generated by infinite sums as well, or is there a step in this construction that allows me to use it?
On this topic of Tensor products of Hilbert spaces, I recommend the book "Fundamentals of the Theory of Operator Algebras, Vol. I" by Kadison and Ringrose (1983) - Chapter 2.6 (p.125 ff.).
As you stated in your comment, tensor products of infinite-dimensional Hilbert spaces can't be Hilbert spaces and at the same time satisfy the defining algebraic property of the tensor product. To circumvent this - without wanting to go into too much detail - for infinite dimensional Hilbert spaces $\mathcal H_1,\mathcal H_2,\mathcal K$ one first introduces the concept of weak Hilbert-Schmidt functionals from $\mathcal H_1\times\mathcal H_2$ to $\mathbb C$ and weak Hilbert-Schmidt mappings from $\mathcal H_1\times\mathcal H_2$ to $\mathcal K$. By construction those are multilinear and bounded, so in particular, jointly continuous. Then, one gets the following result which very much looks analogous to its original algebraic counterpart, refer to Theorem 2.6.4 in the Kadison-Ringrose book.
In this approach, it is quite easy to see that if $(e_i)_{i\in I}$, $(f_j)_{j\in J}$ are orthonormal bases of $\mathcal H_1$, $\mathcal H_2$, respectively, then $(e_i\otimes f_j)_{i\in I,j\in J}$ is an orthonormal basis of $\mathcal H_1\overline\otimes\mathcal H_2$. Of course one might ask further how this concept and the algebraic tensor product $\mathcal H_1\otimes_\text{alg}\mathcal H_2$ are related. One then finds that for Hilbert spaces $\mathcal H_1,\mathcal H_2,\mathcal H$ and a bilinear mapping $\Phi:\mathcal H_1\times\mathcal H_2\to\mathcal H$, the following statements are equivalent.