I'm wondering if there is a way to understand the Hilbert norm-residue symbols without any of the commonly required background (p-adic numbers, local/global fields, class field theory...)
More explicitly, I'm asking if the function $$(\cdot,\cdot)_p:\mathbb{Z}\times\mathbb{Z}\longrightarrow\lbrace\pm 1\rbrace$$ defined as $$(a,b)_p=+1\quad\text{if }ax^2+by^2=z^2\text{ has a non-trivial solution in }\mathbb{Z}/p\mathbb{Z}$$ $$\text{and }(a,b)_p=-1\text{ otherwise}$$ can be proven to have the following properties:
- Symmetric, that is $(a,b)_p=(b,a)_p$
- Bi-multiplicative, that is $(aa',b)_p=(a,b)_p(a',b)_p$
- If $ab$ and $b$ are not divisible by $p$ then $(a,b)_p=+1$ and $(a,p)_p$ is the usual Legendre symbol
On the other hand, can we prove that any functions satisfying these three properties is precisely the function $(\cdot,\cdot)_p$? Actually, I only need that this function exists
I'm trying to understand this part

of the paper "The nonexistence of certain finite projective planes" by Bruck and Ryser, but without going too deep into other stuff
Thanks in advance!!
Your definition of the Hilbert symbol is incorrect if you want it to mean what everyone else understands by the term "Hilbert symbol". The standard meaning of $(a,b)_p$ depends on solvability of $ax^2 + by^2 = z^2$ in the $p$-adic numbers, not in the field $\mathbf Z/p\mathbf Z$.
Example. For a prime $p$, the mod $p$ congruence $px^2 + py^2 \equiv z^2 \bmod p$ has the nontrivial solution $(x,y,z) = (1,1,0)$, so the meaning you would like $(p,p)_p$ to have is $1$, but the equation $px^2 + py^2 = z^2$ has no solution in $\mathbf Q_p$ other than the trivial solution $(0,0,0)$ when $p \equiv 3 \bmod 4$, so the meaning everyone else has for $(p,p)_p$ is $-1$.
The properties you ask your symbol to have are missing some that are listed on the page that you scanned, such as $(-n,n)_p = 1$ in equation (4). Equation (8) says $(n,n+1)_p = (-1,n+1)_p$ for $n > 0$. The reasoning used for (8) applies to $n < 0$ too, so $(n,n+1)_p = (-1,n+1)_p$ for all nonzero integers $n$. (Note an oversight: you only want your symbol defined on nonzero integers, not allowing $0$ as an input.) Since the symbol has values $\pm 1$, the equation $(n,n+1)_p = (-1,n+1)_p$ is equivalent to $(-n,1+n)_p = 1$, and writing $a$ for $-n$ tells us $(a,1-a)_p = 1$ for all nonzero integers $a$. Since your symbol is bimultiplicative, it can be extended uniquely from a mapping $(\mathbf Z - \{0\}) \times (\mathbf Z - \{0\}) \to \{\pm 1\}$ to a bimultiplicative mapping $\mathbf Q^\times \times \mathbf Q^\times \to \{\pm 1\}$. At this point look up the term "Steinberg symbol".
While you do not need to know about local fields in general or class field theory in order to understand Hilbert symbols, you do need to know about $p$-adic numbers. The Hilbert symbol is discussed in Serre's Course in Arithmetic and Cassels' Rational Quadratic Forms without using anything about general local fields or class field theory, but both rely on the $p$-adic numbers. The $p$-adic numbers and $\mathbf Z/p\mathbf Z$ are closely related, but you can't just replace one with the other and expect everything is going to work in the same way. The Hilbert symbol is an example of that: it is about solving a certain polynomial equation in $\mathbf Q_p$, not solving a similar-looking congruence $\mathbf Z/p\mathbf Z$.
Here is another source of difficulty in what you are trying to do. There is a connection between the possibility of having $(a,b)_p = -1$ and the existence of a noncommutative $4$-dimensional division ring over $\mathbf Q_p$ (there is one such ring, analogous to Hamilton's quaternions over $\mathbf R$). But there is no such thing as a $4$-dimensional noncommutative division ring over $\mathbf Z/p\mathbf Z$. In fancier terms, the Brauer group of $\mathbf Q_p$ has an element of order $2$ while the Brauer group of $\mathbf Z/p\mathbf Z$ is trivial.
Remark. Hilbert defined the Hilbert symbol before $p$-adic numbers were created (by Hensel), but that does not mean Hilbert symbols have a simple description that avoids $p$-adic numbers: Hilbert's own definition of his symbol was in terms of being able to solve certain infinite systems of congruences modulo arbitrary powers of $p$ (not just modulo $p$), and part of the point of $p$-adic numbers is that they offer a systematic way of describing congruences modulo arbitrary powers of $p$ as a single equation in one number system. That's why the standard definition of the Hilbert symbol in terms of $p$-adic numbers is simpler than the way Hilbert described it.