In a $R$-module if $a,b\in R$ and $a\neq0, $$b \neq 0$ can it be that $ab=0$? All the modules I have seen do not allow this so I wonder if it is the case generally.
The only axiom that I see which could prevent this is (when $x$ is a vector):
$$(a\cdot_Rb)x=a\cdot_R(bx)$$
Which is odd, since any vector multiplied by elements would have to equal zero, since $0\cdot x=0$. I can't come up with a proof so perhaps someone here can help.
On
You seem to have the ring and the module a bit mixed up in your question: are you interested in $R$ itself as an $R$-module or a more general $R$-module $M$ (say)? In the latter case, if you take $R = \Bbb{Z}$ and $M = \Bbb{Z}/2\Bbb{Z}$, then you have $2x = 0$ for any $x \in M$. If you want $M$ to be $R$ itself, then $R = \Bbb{Z}/m\Bbb{Z}$ provides another example for any non-prime non-unit $m$ (because if $m = ij$ is a non-trivial factorisation of $m$, the product $[i][j]$ of the equivalence classes of $i$ and $j$ will be the equivalence class $[m] = 0$ in $R$).
In a ring $R$, an element $a \in R$ such that there is a $b \in R \setminus \{0\}$ with $ab = 0$ is called a zero divisor. Many rings have zero divisors. Rings that do not have zero divisors are called integral domains.
In an $R$-module $M$, given an element $x$, the set of $a \in R$ such that $ax = 0$ comprise an ideal called the annihilator of $M$. Unless $R = 0$ or $R$ is a field, there will exist non-trivial $R$-modules with non-trivial annihilators. (Because if $I$ is an ideal of $R$ other than $0$ or $R$, $R/I$ is a non-trivial $R$-module with $I$ as its annihilator.)
I note that general elements of a module do not have a multiplication, so $a \cdot b$ need not make sense there.
What you have actually asked is whether two elements of the ring $R$ can have a product which is zero. Yes. In the ring $\mathbb{Z}/6\mathbb{Z}$, $2 \neq 0$, $3 \neq 0$, and $2 \cdot 3 \cong 6 \cong 0$. We say that $\mathbb{Z}/6\mathbb{Z}$ has zero divisors. We can treat $\mathbb{Z}/6\mathbb{Z}$ as a module over itself, yielding a module which does what you are describing.
Normally, the definition of module does not exclude rings with zero divisors. If we wished to exclude zero divisors, one way is to say "module over an integral domain". (This is a restriction of "module" = "module over a ring", in that an integral domain is a commutative ring and it has no zero divisors.)
Note that in any ring with zero divisor, $z$, and (at least) a unit, $u$, there is a $z'$ such that $z z' = 0$. But then $(z+u)z' = zz'+uz' = uz'$, so right cancellation doesn't hold. Be a little careful when there are zero divisors around.