We can define a free module $F$ over a commutative ring with identity $R$ as a free object (with universal property) in the category of unitary $R$-modules. This is equivalent to the existence of a linearly independent generating set for $F$, called a basis as well as $F$ is a coproduct of copies of $R$. It is well-known in the literature that these three definitions are not equivalent, for non-unitary $R$-modules (e.g. Algebra, Hungerford, section 4.2). Can anyone give an idea or a reference for nontrivial counterexamples of these (especially, in the case that free objects need not have a linearly independent generating set for non-unitary modules with non-zero scalar multiplication)? Thanks in advance.
An additional explanation for the basis: by a basis $X$ for $F$ we mean a linearly independent generating set, that is, every non-zero element of $F$ is an element of $X$ or can be uniquely expressed as a linear combination of elements of $X$ with coefficients in $R$ (since $F$ is not unital we cannot necessarily write $x=1.x$ as a linear combination but we may have $x=r.x,$ for some $r\in R$! So any $x\in X$ may be written as a combination of elements of $X$! Thus we should consider the additional assumption for elements of $X$ in $F.$ Of course it will make things complicated.
Literally any nonzero unital ring gives a counterexample: if $R$ is a nonzero unital ring, then $R$ is not a free object in the category of non-unitary $R$-modules (even though it is a direct sum of copies of $R$ and has a linearly independent generating set). The reason is simple: if $f:R\to M$ is any module homomorphism, then the image of $f$ must be a unitary submodule of $M$, since $R$ is a unitary module. So, $R$ cannot be free, since its free generators cannot be mapped to non-unitary elements of other modules.
In general, non-unitary modules become much easier to understand when you realize they are just the same thing as unitary modules over the unitalization $U(R)$ of $R$. Here $U(R)$ is the ring obtained by freely adjoining a unit to $R$. Concretely, $U(R)=\mathbb{Z}\oplus R$ with multiplication defined by $(a,r)\cdot(b,s)=(ab,as+br+rs)$ (the idea being that $(1,0)$ is the unit so $(a,r)$ is the formal sum $a\cdot 1+r$). A non-unitary $R$-module can then be thought of as a unitary $U(R)$-module where you define $(a,r)\cdot m$ as $am+rm$. So free objects in the category of non-unitary $R$-modules are just ordinary free $U(R)$-modules--that is, direct sums of copies of $U(R)$.