Let $f:X\to \operatorname{Spec}\mathbf{Z}_p$ be a smooth proper $\mathbf{Z}_p$-scheme and $F$ a coherent sheaf on $X$ which is flat over $\mathbf{Z}_p$. Further suppose that $H^1(X_p, F_p)=0$ where $X_p$ denotes the special fiber $X\otimes_{\mathbf{Z}_p} \mathbf{F}_p$ of $X$ and $F_p$ the pullback of $F$ to $X_p$.
Question 1: Is it then true that $R^1f_*F$ has empty support?
Question 2: What has this to do with the torsion of (the global sections) of $F$? Can we conclude from this that $H^0(X,F)$ is a free $\mathbf{Z}_p$-module?
This seems all to follow from statements in Chapter III, Section 12 of Hartshorne but I fail to understand what the meaning of the higher direct images $R^if_*F$ really is.
EDIT: I also happen to now that there is a short exact sequence
$0\to H^0(X,F')\to H^0(X,F'')\to H^0(X,F)\to 0$
and that the first two terms commute with base change by $\mathbf{F}_p$, i.e.
$H^0(X,F')\otimes F_p = H^0(X_p,F'_p)$ and the same is true for $F''$.
The proof that I am trying to understand is the proof of Lemma 1.1 here.
$\newcommand{\Spec}{\mathrm{Spec}}$$\newcommand{\Z}{\mathbb{Z}}$$\newcommand{\Q}{\mathbb{Q}}$I don't have Hartshorne on me, but I assume that the cited section is on proper base change/semi-continuity.
Since $f:X\to \Spec(\Z_p)$ is proper, $\mathcal{F}$ is $\Z_p$-flat and coherent, and $S$ and $X$ are Noetherian then we know that the map $f_i:\Spec(\Z_p)\to \Z$ given by $s\mapsto \dim_{k(s)}H^i(X_{k(s)},\mathcal{F}_{k(s)})$ is upper semi-continuous for all $i\geqslant 0$. In particular since $f_1$ is upper semi-continuous we know that $f^{-1}((-\infty,1))$ is an open subset of $\Spec(\mathbb{Z}_p)$. Since you're assuming that $p\in f^{-1}((-1,\infty))$ you can deduce that $\Spec(\Z_p)=f^{-1}((-\infty,1))$ since the only open subset of $\Spec(\Z_p)$ containing $p$ is $\Spec(\Z_p)$. Thus, $H^1(X_{\Q_p},\mathcal{F}_{\Q_p})=0$.
Now, since $\Spec(\Z_p)$ is connected and reduced we can use the same theory to show that since $f_1$ is constant that $R^1 f_\ast\mathcal{F}$ is locally free and, moreover, that we have an isomorphism $(R^1 f_\ast\mathcal{F})_{k(s)}\cong H^1(X_{k(s)},\mathcal{F}_{k(s)})=0$ for all $s$ so that $R^1 f_\ast\mathcal{F}=0$.
EDIT: To answer your other question, it is true that $H^0(X,\mathcal{F})$ is a free $\Z_p$-module. This is more sophisticated though (I don't think it's in Hartshorne?) but it's a classical result that follows from things written in Mumford's book on abelian varieties. I found a reference here in the form of Corollary 2.3.
Indeed, since $\Spec(\Z_p)$ is affine we know that $f_\ast \mathcal{F}=\widetilde{H^0(X,\mathcal{F})}$ (see the second edit below) and so the claim you want to prove is that $f_\ast \mathcal{F}$ is locally free. This is then what's stated in the cited notes of Osserman.
EDIT EDIT: Just to point out that in your response to "I don't understand what $R^i f_\ast\mathcal{F}$ is really" note that if you have a reasonable map (qcqs probably?) $f:X\to\Spec(A)$ then $H^i(X,\mathcal{F})$ is an $A$-module and $R^if_\ast\mathcal{F}$ is just the quasi-coherent sheaf $\widetilde{H^i(X,\mathcal{F})}$. All of the above is then by way of asking questions like "Is it true that if $H^1(X,\mathcal{F})\otimes_{\mathbb{Z}_p}\mathbb{F}_p=0$ that $H^1(X,\mathcal{F})=0$? Moreover, in this case, is it true then that $H^0(X,\mathcal{F})$ is a free $\mathbb{Z}_p$-module?" The answers to both of these questions being yes by the above.