It is well known that integers can be represented as a sum of squares, two, three and more. In what follows will be given a way to represent integers as a sum of 3 squares using triangular numbers.
It is also well known that adding 2 consecutive triangular numbers produces a square. So if we wanted to get 3 squares, we just need to add 3 pairs of consecutive triangular numbers. Here we consider the general case of $$N = a^2 + b^2 + c^2$$ with a,b,c not consecutive and not equal. In practice, some numbers will have a representation of 3 squares that are consecutive and sometimes equal.
For that we write N as $$N = (T_{n} + T_{n+1}) + (T_{n+k} +T_{n+k+1}) + (T_{n+j} + T_{n+j+1})$$
where the T's are triangular numbers and n,n+k,...are their indices. Using the formula for a triangular number $$T_{n}=n(n+1)/2$$ we transform the previous equation into the following one: $$N = 3n^2 + (2k+2j+6)n + 1 + (k+1)^2 + (j+1)^2$$
This is nothing but a quadratic equation in n and will be solved by looking for values of j and k that will make its discriminant $$d = b^2-4ac$$ a square. If we take the example of N=77, the discriminant $$d = 231 + 2kj -2(k^2+j^2)$$ will be a square for two pairs of values for $$(k,j)=(1,2),(1,6)$$ and this will give a value for $$n = (3,1)$$
Now we have the values of the indices (n,k,j), we can use the corresponding triangular numbers to form the squares and we get: $$ N = 77 = 2^2 + 3^2 + 8^2 = 4^2 + 5^2 + 6^2$$
If we look at the discriminant d, we see that the potential squares will be $$s^2=15^2, 14^2, 13^2...$$ So we can either try different values of (j,k) that will make d a square or solve another quadratic in j or k given by: $$2kj - 2(k^2 +j^2) = -6$$ since $$231 = 15^2 + 6$$
The question is when do we know that we have found all the 3 squares representations of a given number N?
It will be very inefficient to check every potential square value of the discriminant d unless we wanted to find all the representations.