Let $K$ be a number field and $\mathcal{O}_K$ be the ring of integers. Let $\overline{\mathbb{F}_{p}}$ be the algebraic closure of the finite field with $p$-elements. What are the points $\operatorname{Spec} \mathcal{O}_{K} (\overline{\mathbb{F}_p})$?
I have seen in the past the definition of the $R$-points of a scheme defined over $R$ as sections of the structure map, but in this case I don't see how $\operatorname{Spec} (\mathcal{O}_K)$ is naturally a scheme over $\overline{\mathbb{F}_p}$. My best guess is that we take the sections of the natural map $\operatorname{Spec}(\mathcal{O}_K) \times_{\operatorname{Spec} (\mathbb{Z})} \operatorname{Spec}(\overline{\mathbb{F}_p}) \to \operatorname{Spec} (\overline{\mathbb{F}_p})$, but I want to be sure.
You're right, $\operatorname{Spec}(\mathcal O_K)$ is not a scheme over $\overline{\mathbb F}_p$, if it was you'd have a morphism $\operatorname{Spec}(\mathcal O_K)\to\operatorname{Spec}(\overline{\mathbb F}_p)$ corresponding to a morphism $\overline{\mathbb F}_p\to\mathcal O_K$ which doesn't exist.
Here is a general definition: if $X$ and $T$ are schemes then an $T$-point of $X$ is just a morphism $T\to X$. If there is an understood base scheme $S$ you are working over then you take $S$-morphisms $T\to X$.
Note this coincides with the definition of $R$-point you have given, in the particular case where $T=\operatorname{Spec}(R)$ and you are working over the base $S=\operatorname{Spec}(R)$, as a $\operatorname{Spec}(R)$-point of $X$ (usually just called an $R$-point of $X$) is an $R$-morphism $\operatorname{Spec}(R)\to X$, which is the same as a section of the structure map.
With this general definition you can work out what an $\overline{\mathbb F}_p$-point is of $\operatorname{Spec}(\mathcal O_K)$.