I think I understand the definition of a free $R$-module over a set $X$: it is given by the set of all maps from $X$ to $R$ which vanish at all but finitely many points of $X$. The module operations are pointwise addition and scalar multiplication of functions. The free module over $X$ has the set $\{ \chi_p:p\in X\}$ as generators, where $\chi_p: X\rightarrow R$ has value one at $p$ and vanishes otherwise. Given the identification $\chi_p \leftrightarrow p$, the free $R$-module over $X$ can also be described as formal finite linear combination of elements of $X$ with coefficients in $R$.
So far so good, but I do not understand the need for the finitely many condition. Surely the set of all functions from $X$ to $R$ is an $R$-module. I imagine it fails to be free, but I am not sure about why. For example if a function $f$ vanish at all but a countable number of points $\{p_i\}, i\in I$, I do not see why it could not be written as $f=\sum_{i\in I} f(p_i)\chi_{p_i}$. Wouldn't then $\{\chi_p:p\in X\}$ still be a basis?
Linear combinations are themselves by definition so that almost all summands are zero. For an algebraist the sum of infinitely many nonzero summands is not really a sum.
Also, the infinite variant $R^X$ is certainly not free over the given set. As such it would have to be true that for any other module $A$ and map $f\colon X\to A$ there exists a unique module homorphism $h\colon R^X\to A$ such that $h\circ \operatorname{inc}=f$. Now let $A$ be the "true" (i.e., finite sum based) free module and $f$ the inclusion. Then there is either no or not a unique such homomorphism.