I would like to find a language $\mathcal{L}$ and first order sentence $\phi$ of $\mathcal{L}$ so that its finite spectrum is
$\{p^n ~:~ n > 0, \text{ p is prime}\}$
$\{ p ~:~ p \text{ is prime}\}$
It seems like a large library of mathematical knowledge is needed to even begin consider such questions, and I have to admit mine are thin.
HINT: For the first problem consider finite fields.
The trick for the second problem is to construct a language and sentence for which the finite models look enough like initial segments of the positive integers, or something very similar. One especially elegant approach uses the something very similar idea. You’ll want a language $\{+,\times,\le,',2,a,b\}$. Then write down axioms that say that
Show that if $2\le n\in\Bbb Z^+$ there is (up to isomorphism) exactly one model of these axioms of cardinality $n$, and it’s $\{2,\ldots,n\}$ with the usual ordering and operations, with $a$ interpreted as $n$ and $b$ as $n+1$, and all of the integers greater than $n$ collapsed into the single entity $b$.
Note that an integer $n>1$ is composite iff it’s the product of two smaller integers greater than $1$, so the compositeness (and hence the primality) of $a$ can be expressed in this language.