I have looked at five algebraic topology books, but the only computation of cohomology ring of $\mathbb{RP}^\infty$ which I could found is with $\mathbb{Z}_2$-coefficient : $$ H^*(\mathbb{RP}^\infty, \mathbb{Z}_2) \cong \mathbb{Z}_2[\alpha] $$ where $\alpha$ generator of degree 1.
From this computation, using the homomorphismes $\mathbb{Z} \to \mathbb{Z}_2$ or $\mathbb{Z}_{2k} \to \mathbb{Z}_2$, looking at induced map of chain complexes, I believe we can compute cohomology with coefficient in $\mathbb{Z}$ or $\mathbb{Z}_{2k}$.
However, the question : how to compute cohomology ring of real projective space with $\mathbb{Z}_p$ coefficient? We have no such morphism of groups.
For odd $p$, one has $H^*(\Bbb RP^\infty; \Bbb Z_p)=0$.
Consider the cellular cochain complex of $\Bbb RP^\infty$ with $\Bbb Z_p$-coefficients:
$$\cdots \stackrel{\cdot 2}{\leftarrow} \Bbb Z_p \stackrel{0}{\leftarrow} \Bbb Z_p \stackrel{\cdot 2}{\leftarrow} \Bbb Z_p\stackrel{0}{\leftarrow} \Bbb Z_p\leftarrow 0.$$
Note that $2\Bbb Z_p = \Bbb Z_p$ and $\ker(\Bbb Z_p \stackrel{\cdot 2}{\leftarrow}\Bbb Z_p)=0$, hence all cohomology groups vanish.