What is the cohomology ring $$ H^*(\mathbb{R}P^\infty;\mathbb{Z})?$$ $$ H^*(\mathbb{R}P^n;\mathbb{Z})?$$
for mod 2 coefficient, the answer is on Hatcher's book and Proving that the cohomology ring of $\mathbb{R}P^n$ is isomorphic to $\mathbb{Z}_{2}[x]/(x)^{n+1}$. For the graded module structure, it is obtained from the homology struture.
As pointed out by fixedp in the comments, Hatcher computes $H^*(\mathbb{RP}^{\infty}; \mathbb{Z})$ after the proof of Theorem $3.19$. The result is
$$H^*(\mathbb{RP}^{\infty}; \mathbb{Z}) \cong \mathbb{Z}[\alpha]/(2\alpha),\quad |\alpha| = 2.$$
He then says the same technique can be used to show
\begin{align*} H^*(\mathbb{RP}^{2k}; \mathbb{Z}) &\cong \mathbb{Z}[\alpha]/(2\alpha, \alpha^{k+1}), \quad |\alpha| = 2\\ H^*(\mathbb{RP}^{2k+1}; \mathbb{Z}) &\cong \mathbb{Z}[\alpha, \beta]/(2\alpha, \alpha^{k+1}, \beta^2, \alpha\beta),\quad |\alpha| = 2, |\beta| = 2k + 1. \end{align*}