Let $E$ be a nonempty set and $R$ a ring. Define $$R^{(E)} = \oplus_{e\in E}R,$$ in other words the submodule of the product $\prod_{e\in E}R$ that consists of the elements that have only finitely many nonzero components. We embed $E$ into $R^{(E)}$ by identifying $e\in E$ with $R^{(E)}$ having $e^\mathrm{th}$ component the unity of $R$ and all other components zero.
Then my source (Wikipedia) claims that each element of $r\in R^{(E)}$ can be written uniquely as $r=\sum_{e\in E}c_e e$ where only finitely many $c_e$ are nonzero. How would one go about proving it formally? A priori it does not seem to me that this would be the case.
To that you can so express them, note that by definition you can find a finite set $F\subseteq E$ such that $\pi_e(r)=0$ for all $e\notin F$ (that is, a finite set that contains the support of $r$. Thus, using the given notation, we can express $$r = \sum_{e\in F}\pi_e(r)e.$$ We use $F$ to ensure the sum is finite; if you are willing to abuse notation, you could simply write $r=\sum_{e\in E}\pi_e(r)e$, with the knowledge that all but finitely many of the terms are equal to $0$ so the “infinite sum” is really a finite sum.
As to uniqueness, note that this is a submodule of the direct product. Each element $x$ of the direct product is uniquely determined by the set of its images under the projection maps, $(\pi_e(x))_{e\in E}$. Thus, the expression is unique because it is based on its expression as an element of the direct product.