Let $X$ be a topological space such that all the singular homology groups $H_n(X)$ are finitely generated free abelian. Then by the universal coefficient theorem in cohomology, we have $H^n(X;R)\approx \text{Hom}(H_n(X);R) \approx \prod_i R=\bigoplus_i R$ (by finite generateness where $H_n(X)\approx \bigoplus_i \Bbb Z)$ for any ring $R$. Consider the special cases $R=\Bbb Z$ and $R=\Bbb Z_p=\Bbb Z/p\Bbb Z$. Since tensor products commute with direct sums, the above observation shows that $$ H^n(X;\Bbb Z)\otimes \Bbb Z_p \approx (\bigoplus_i \Bbb Z)\otimes \Bbb Z_p \approx \bigoplus_i(\Bbb Z\otimes \Bbb Z_p)\approx \bigoplus _i \Bbb Z_p\approx H^n(X;\Bbb Z_p). $$ On the other hand, we can directly define a map $H^n(X;\Bbb Z)\otimes \Bbb Z_p\to H^n(X;\Bbb Z_p)$ by $[f]\otimes a \mapsto [\bar{f}]$ where $f$ is a cocycle $C_n(X)\to \Bbb Z$ and $\bar{f}$ is the composition of $f$ with the canonical map $\Bbb Z\to \Bbb Z_p$.
Now my question is, is the above isomorphism $H^n(X;\Bbb Z)\otimes \Bbb Z_p \to H^n(X;\Bbb Z_p)$ is in fact given by the map $[f]\otimes a \mapsto [\bar{f}]$? (I am hoping this to hold)
Even if not, is the map $[f]\otimes a \mapsto [\bar{f}]$ an isomorphism?
The isomorphism you describe depends on a choice of generators for your cohomology. So the question of "Are these two maps the same?" is ill-posed since one map depends on choices.
The map you describe is indeed an isomorphism. We can check surjectivity as follows: suppose $[\sigma] \in H^n(X; \mathbb{Z}/p)$ we wish to show that it is the reduction of some $[\alpha] \in H^n(X; \mathbb{Z})$. Since the homology is free abelian, the universal coefficient theorem tells us that $[\alpha]$ is the same as some $f \in \operatorname{Hom}(H_n(X),\mathbb{Z})$, and that $[\sigma]$ is the same as some $g \in \operatorname{Hom}(H_n(X),\mathbb{Z}/p)$. So the question of surjectivity is the same as the question of if the map given by $f \rightarrow \bar{f}$ (where $\bar{f}$ is reduction mod p), is surjective. This map is surjective since any map from a free abelian group to $\mathbb{Z}/p$ factors through $\mathbb{Z}$.
Then injectivity follows from a dimension count.