Prove that for $p\ge 3$, a prime: $$\left(\frac{-2}{p}\right) = \begin{cases} 1 & p\equiv1,3\pmod{8}\\ -1 & p\equiv-1,-3\pmod{8} \end{cases}$$
I already now that:
$$\left(\frac{-2}{p}\right)=\left(\frac{-1}{p}\right)\left(\frac{2}{p}\right)=(-1)^{\frac{p-1}{2}}(-1)^{\frac{p^{2}-1}{8}}=(-1)^{\frac{(p-1)(p^{2}-1)}{8}}=(-1)^{\frac{p-1}{2}\cdot\frac{p+1}{2}\cdot\frac{p-1}{4}}$$
And I'm trying to show when the exponent is even (for the $1$ case).
We have:
- $\frac{p-1}{4} = 2k \implies p- 1 = 8k \implies p = 8k + 1$. So we can infer that if $p\equiv 1\pmod{8}$ then $\left( \frac{-2}{p} \right)=1$
- $\frac{p-1}{2}$ - doesn't yield anything new to us (I think)
- $\frac{p+1}{2} = 2t \implies p+1 = 4t \implies p = 4t-1$
but I don't really see how to infer the other option $(p \equiv 3 \pmod{8})$
There's an error in your computation: $$\binom{-2}p=(-1)^{\tfrac{p-1}{2}}(-1)^{\tfrac{p^{2}-1}{8}}=(-1)^{\tfrac{(p-1)\color{red}{+}(p^{2}-1)}{8}}=(-1)^{\tfrac{p-1}{2}\bigl(1+\tfrac{p+1}{4}\bigr)}=(-1)^{\tfrac{p-1}{2}\tfrac{p+5}{4}}$$
This is equal to $1$, i.e. $-2$ is a square mod. $p$, if and only if $\;\dfrac12\dfrac{p-1}2\,\dfrac{p+5}2\,$ is even. Observe that $$\dfrac{p-1}2\;\text{is}\:\begin{cases}\text{even}\\\text{odd}\end{cases}\iff \dfrac{p+5}2\;\text{is}\:\begin{cases}\text{odd}\\\text{even}\end{cases}$$ so we have the following cases: