Defining an $A$-algebra structure using a basis and "constants of structure"

Question

Let $A$ be a commutative ring and $E$ an $A$-module with a basis $(e_i)_{i\in I}$. Let $(\alpha_{ijk})_{(i,j,k)\in I\times I\times I}$ be a family of elements of $A$ such that, for $i,j\in I$, the subfamily $(\alpha_{ijk})_{k\in I}$ is of finite support. We want to define a $A$-bilinear mapping from $E\times E$ to $E$ (i.e. a "multiplication"). To that end, write $$e_ie_j:=\sum_{k\in I}\alpha_{ijk}e_k$$ for every $i,j\in I$. I can't see how this defines a bilinear map. For example, let $i,j\in I$ and $\beta\in A$. Then, clearly $\beta(e_ie_j)=\sum_{k\in I}(\beta\alpha_{ijk})e_k$. I need to show that $\beta(e_ie_j)=(\beta e_i)e_j=e_i(\beta e_j)$. But what does $(\beta e_i)e_j$ equal to? The multiplication is defined only for pairs of basis elements, so how does one multiply $(\beta e_i)e_j$? Or even $e_i(\beta e_j)?$