I am new to etale cohomology theory and recently learnt some base change theorems like proper base change and smooth base change. However, I am somehow confused about the etale cohomology group of $\Bbb Z/p\Bbb Z$ after base change.
For example, asssume we have a scheme $X$ proper smooth over $\Bbb Z_p$, where $\Bbb Z_p$ means the $p$-adic integer ring. What is the relation between $H^i_{et}(X_{\Bbb F_p},\Bbb Z/p\Bbb Z)$ and $H^i(X_{\Bbb Q_p},\Bbb Z/ p\Bbb Z)$? Must them have the same dimension as $\Bbb F_p$-vector space?
The answer to the question, as written, is no. Let me give you a silly reason it can never be true, and then give you a more interesting way of constructing a concrete example.
Note that if $X/\mathbb{F}_p$ is a smooth (not needed, but makes me feel better) geometrically connected variety of dimension $n$, then $H^i(X,\mathbb{F}_p)=0$ for $i>\dim X$. Why?
Define a map of etale sheaves $a:\mathcal{O}_X\to \mathcal{O}_X$ given by $x\mapsto x^p-x$. I claim that this is a group map (clear) and that it's surjective. To see that it's surjective, we merely note that if we have some $U\to X$ in $Et(X)$ and some $s\in O_X(U)$. We need to find an etale cover of $U$ where, upon restriction to $U$, $s$ is hit by our map. The obvious candidate is $U':=\mathrm{Spec}(\mathcal{O}_U[t]/(t^p-t-s))$. The only question is whether $U'\to U$ is etale, but this is easy to check (since the deriative of $t^p-t-s$ is $-1$). So, our map is surjective, what is its kernel? I claim that it's naturally isomorphic to $\underline{\mathbb{Z}/p\mathbb{Z}}$.
To see this, it suffices to show that for any connected object $U\to X$ in $Et(X)$ we have that $\ker(a)(U)$ is isomorphic to $\mathbb{Z}/p\mathbb{Z}$. Note though that since $X$ is smooth, so is $U$. Since $U$ is smooth and connected it's integral. Thus $O_X(U)$ is an integral domain. Since $O_X(U)$ is an integral domain, the solutions to $x^p-x=0$ can be at most $p$ in number, and since $\mathbb{F}_p$ fits that bill, we see that $\mathbb{F}_p$ is everything. Thus, we've actually shown that $\ker(a)$ should be thought of as $\underline{\mathbb{F}_p}$ but, since we're only interested in the underlying group structure, $\underline{\mathbb{Z}/p\mathbb{Z}}$ is a better way to write it.
So, we have a short exact sequence of abelian etale sheaves
$$0\to \underline{\mathbb{Z}/p\mathbb{Z}}\to \mathcal{O}_X\to\mathcal{O}_X\to 0$$
and so, of course, we get a long exact sequence in cohomology groups. The claim then follows from Grothendieck vanishing for coherent sheaves recalling that that the cohomology of a coherent sheaf is the same on either the etale or Zarisk site.
Thus, for $\mathbb{Z}/p\mathbb{Z}$, you can essentially NEVER have equality for all $i$.
Let's now find an example to show that it doesn't even work for $i=1$. As per usual, the easiest place to look for interesting, doable examples is abelian varieties. Specifically, if $k$ is a field and $A/k$ is an abelian variety then $H^1_\mathrm{\acute{e}t}(A,\mathbb{Z}/p\mathbb{Z})$ can be identified, as a $\mathbb{Z}/p\mathbb{Z}[\mathrm{Gal}(\overline{k}/k)]$-module, with the dual of the $\mathbb{Z}/p\mathbb{Z}[\mathrm{Gal}(\overline{k}/k)]$-module $E[p](\overline{k})$.
In particular, if $\mathscr{A}$ is an abelian variety over $\mathbb{Z}_p$, then what you're asking is whether or not the $\mathbb{Z}/p\mathbb{Z}$-modules $\mathscr{A}[p](\overline{\mathbb{Q}_p})$ and $\mathscr{A}[p](\overline{\mathbb{F}_p})$ have the same cardinality. This is never the case.
In fact, as you are probably well-aware, since $\mathbb{Q}_p$ is of characteristic $0$, then $\mathscr{A}[p](\overline{\mathbb{Q}_p})$ always as $p^{2g}$ elements, where $g=\dim(\mathscr{A})$. That said, if $A$ is an abelian variety over $\mathbb{F}_p$ (e.g. $\mathscr{A}_{\mathbb{F}_p}$) then $A[p](\overline{\mathbb{F}_p})$ always contains strictly less than $p^{2g}$ elements. There are several ways to prove this, but the essential idea is that the multiplication by $p$ map $[p]:A\to A$ is finite of degree $p^{2g}$. Thus, to say that $A[p](\overline{F_p})$ has $p^{2g}$-points means that the group scheme $A[p]$ is etale or, equivalently, that the map $[p]$ is etale.
This is certainly not the case because the derivative of this map is $0$. The issue, as you might expect, is Frobenius. Namely, (since we're over $\mathbb{F}_p$ we don't have to worry about Frobenius twists) the Frobenius map $F:A\to A$ is a group map and so, in particular, $\ker F$ is a subgroup scheme of $A$. Note though that the degree of the finite map $F$ is $p^g$, and thus we see that $\ker F$ is a group scheme of order $p^g$. In particular, $\ker F$ is $p^g$-torsion, and so $\ker F\subseteq A[p^g]$. If $A[p]$ were etale, then so would $A[p^g]$ since it's an iterated extension of $A[p]$. But, $\ker F$ is clearly not etale.
Thus, any abelian variety gives a counterexample in the case when $i=1$. To be concrete, we can look at elliptic curves, in which case supersingular elliptic curves $\mathcal{E}$ over $\mathbb{Z}_pp$ (e.g. $y^2=x^3+1$ when $p=2\mod 3$) have the property that $H^1(\mathcal{E}_{\mathbb{Q}_p},\mathbb{Z}/p\mathbb{Z})$ are (non-canonically) isomorphic to $(\mathbb{Z}/p\mathbb{Z})^2$ but $H^1(\mathcal{E}_{\mathbb{F}_p},\mathbb{Z}/p\mathbb{Z})$ is $0$.
Of course, as you probably know, the smooth proper base change says that they do have the same dimension (and in fact have equivariant Galois actions) if you work with coefficients in $\underline{G}$ where $G$ is a finite abelian group such that $p\nmid |G|$.
Hope this helps!