$\newcommand{\kron}[2]{\left( \frac{#1}{#2} \right)}$ It's easy to use Quadratic Reciprocity to show that $\kron{5}{p} = \kron{p}{5} = 1$ when $p \equiv \pm 1 \pmod 5$, and is $-1$ when $p \equiv \pm 2 \pmod 5$.
I'm interested in an elementary, direct proof of when this happens without appealing to either quadratic reciprocity directly or to things like Gauss's lemma (which is sort of like quadratic reciprocity in disguise). For example, this answer gives a direct proof of $\kron 2p$.
For $p \pmod 5$, this is not so bad. For instance, if $p \equiv 1 \pmod 5$, then one can use that $\left( \mathbb{Z}/p\mathbb{Z} \right)^\times$ is cyclic of order divisible by $5$ to get an element of order $5$, and then proceed as in this question.
But what about when $p \not \equiv 1 \pmod 5$? In particular, how might we handle when $p \equiv -1 \pmod 5$?
Here's a start: If $p \equiv 1$ (mod $5$), then $F = \mathbb{Z}/p\mathbb{Z}$ contains an element of multiplicative order $5$, so $x^{5}-1 = (x-1)(x^{4}+x^{3}+x^{2}+x+1)$ splits into linear factors in $F[x].$ If $\omega \neq 1$ is one of its roots, then $(\omega+\omega^{-1})^{2} = 1- \omega - \omega^{-1}$ so that $x^{2} +x-1$ has a root in $F.$ Then $F$ contains $\sqrt{5}$ by solving the quadratic.
Continuation: Suppose now that $p \equiv -1$ (mod $5$). Then ${\rm GF}(p^{2})$ contains an element $\omega$ of multiplicative order $5$. Note however that $\omega +\omega^{p} \in {\rm GF}(p).$ Also, as $p \equiv -1$ (mod $5$), we have $\omega^{p} = \omega^{-1}.$ Hence $x^{2}+x-1$ has a root in ${\rm GF}(p),$ so that we still have $\sqrt{5} \in {\rm GF}(p).$
I think the other direction follows ( at least for odd $p$) by mimicking the proof of the constructibility of the regular pentagon but working over ${\rm GF}(p).$ More precisely, if ${\rm GF}(p)$ contains $\sqrt{5}$ we can show that a primitive $5$-th root of unity $\omega$ satisfies a quadratic equation over ${\rm GF}(p).$ Hence ${\rm GF}(p^{2})$ contains an element of multiplicative order $5$, so that $5$ divides $p^{2}-1$ and $p \equiv \pm 1$ (mod $5$).
More detail: We know that $ \alpha = \omega +\omega^{-1}$ is a root of $x^{2}+x-1$ as above, and lies in ${\rm GF}(p).$ Now $(\omega - \omega^{-1})^{2} = -2 - (1 + \omega + \omega^{-1}) = -3 - \alpha.$ Hence $\omega - \omega^{-1}$ lies in a quadratic extension (at most) of ${\rm GF}(p).$ Since $\omega+ \omega^{-1} \in {\rm GF}(p),$ we see that $\omega$ itself lies in a (at most) quadratic extension on ${\rm GF}(p).$
To treat the case $p = 2,$ note that $2$ is not a quadratic residue (mod $5$) by inspection.
Later note in response to question of PI: I think this approach may generalize, but it seems to stray into the realm of Gauss sums, and I have not worked through all the details: let $p,q$ be odd primes, and let $\omega$ be a primitive $q$-th root of unity in $\overline{{\rm GF}(p)}.$ Consider $\sigma = \sum_{i=0}^{\frac{q-3}{2}} \omega^{p^{i}}.$ Suppose that $p$ is a quadratic residue (mod $q$). Then $p^{\frac{q-1}{2}} \equiv 1 $ (mod $q$). Hence $\sigma^{p} = \sigma$ and $\sigma \in {\rm GF}(p).$ Conversely, if $p$ is not a quadratic residue (mod $p$), then $\sigma^{p} \neq \sigma$ as $p^{\frac{q-1}{2}} \not \equiv 1 $ (mod $q$).