Let $p \gt 3$ be an odd prime,
$$p \equiv 3 \pmod 4 \iff \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1}k \ \ \ \ + \prod_{1\le k \le p-1\\ \ \left(\frac{k}{p}\right)=-1}k \ \ \ \equiv 0 \pmod {p^2}$$
This seems to be true, but I could not find a proof.
Though it is quite easy to prove that the above congruence holds modulo $p$ for every odd prime, whatever its residue modulo $4$.
ex:
$p=5: \ \ 1\cdot4 +2\cdot 3 = 10 = 2\cdot 5$
$p=7: \ \ 1\cdot4\cdot2 +3\cdot 5 \cdot 6 = 98 = 2\cdot 7^2$
etc.
And I also know how to prove that: $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=\epsilon }k \ \ \ \ \equiv -\epsilon(-1)^{\frac{p-1}{2}} \pmod {p}$$
But from here, I could not find a path to the above congruence modulo $p^2$.
Any other idea?
Please check the following proposed proof (for $\Rightarrow$) .
We have: $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k \ \ \ \ \equiv \prod_{1\le a \le \frac{p-1}{2}\\ \ \ }a^2 \pmod {p}$$ because it is well known that all the squares $\pmod p$ are obtained by squaring all the $a$ such that $1\le a \le \frac{p-1}{2}$. (For, if $b$ is a square root of $a$ with $1\le a,b \le p-1$, so is $p-b$, and either $b$ or $p-b$ must be smaller than $\frac{p-1}{2}$.)
Then, since $(p-1)!+1 \equiv 0\pmod {p}$ (Wilson theorem) and $(\frac{p-1}{2}!)^2+(-1)^\frac{p-1}{2} \equiv 0\pmod {p}$ (see here), we have: $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k \ \ \ \ \equiv -(-1)^{\frac{p-1}{2}} \pmod {p} \ \text{,} \ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1 }k \ \ \ \ \equiv (-1)^{\frac{p-1}{2}} \pmod {p}$$ and
$$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1 }k \ \ \ \ \equiv 0 \pmod {p}$$
Now , suppose $p \equiv 3 \pmod 4 $, $(-1)^{\frac{p-1}{2}}=-1$, then: $$\left(\frac{p-k}{p}\right)=\left(\frac{-k}{p}\right)=\left(\frac{-1}{p}\right)\left(\frac{k}{p}\right)=(-1)^{\frac{p-1}{2}}\left(\frac{k}{p}\right)=-\left(\frac{k}{p}\right)$$ Then, $$\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1 }k =\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k + \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }(p-k)$$ Then, doing the development on the RHS, but dropping the powers of $p$ higher than $p^2$: $$\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1 }k =\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k + (-1)^{\frac{p-1}{2}}\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k-p (-1)^{\frac{p-1}{2}}\sum_{1\le j \le p-1\\ \ \ \left(\frac{j}{p}\right)=1 }\frac{1}{j}\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k+\cdot\cdot\cdot $$ Then: $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1}k \equiv p\cdot \sum_{1\le j \le p-1\\ \ \ \left(\frac{j}{p}\right)=1 }\frac{1}{j}\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k \pmod {p^2} $$
but $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k\ \ \ \ \equiv -(-1)^{\frac{p-1}{2}}=1 \pmod {p} $$ then, $$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1}k \equiv p\cdot \sum_{1\le j \le p-1\\ \ \ \left(\frac{j}{p}\right)=1 }\frac{1}{j}\pmod {p^2} $$ but, by the same argument as at the begining of the proof:
$$\sum_{1\le j\le p-1\\ \ \left(\frac{j}{p} \right)=1}\frac{1}{j} \equiv \sum_{1\le a\le \frac{p-1}{2}}\frac{1}{a^2} \pmod p$$ And, since by Wolstenholme's theorem, if $p>3$, $$ \sum_{1\le a\le p-1}\frac{1}{a} \equiv 0 \pmod {p^2}$$ then , if $p>3$, $$ 0 \equiv \frac{1}{p}\sum_{1\le a\le p-1}\frac{1}{a} = \frac{1}{p}\sum_{1\le a\le \frac{p-1}{2}}\frac{1}{a}+\frac{1}{p}\sum_{1\le a\le \frac{p-1}{2}}\frac{1}{p-a} = \sum_{1\le a\le \frac{p-1}{2}}\frac{1}{a(p-a)} \equiv -\sum_{1\le a\le \frac{p-1}{2}}\frac{1}{a^2} \pmod p $$
then , if $p>3$
$$ \prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1}k \equiv 0 \pmod {p^2} \ \ \ \square $$
A proof for the converse ($\Leftarrow$) is still missing. By numerical search no $p \le 400000$ such that $p \equiv 1 \pmod 3$ satisfy the above congruence. But the quotient $$ \frac{\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=1 }k +\prod_{1\le k \le p-1\\ \ \ \left(\frac{k}{p}\right)=-1}k}{p} \pmod p $$ appears to be randomly and homogeneously distributed between $0$ and $p-1$. Then, such a $p$ might well exist above $400000$ and ($ \Leftarrow$) would be wrong actually.