I'm trying to factor $n=1829$ by hand with a quadratic sieve, using the factor base $B=\{-1,2,3,5,7,11,13\}$. Now, by computing various squares of the form $(\lfloor \sqrt(in) \rfloor + j)^2 \mod n$, I've obtained the following:
\begin{align*} 42^2 &\equiv -65 &&= -1*5*13 \\ 43^2 &\equiv 20 &&= 2^2*5 \\ 61^2 &\equiv 63 &&= 3^2*7 \\ 85^2 &\equiv -91 &&= -1*7*13 \end{align*} Putting these together, I get that $$(42*43*61*85)^2 \equiv (-1*2*3*5*7*13)^2 \mod n$$ and I'm able to proceed with the factorization.
My question is the following: When I originally reduce the squares modulo $n$, how do I know which value to take? For example, I took $42^2 \equiv -65$ instead of the more obvious choice of $1764$. Similarly for $85^2 \equiv -91$. Of course, there are an infinite number of possible choices for which value each of these can take. How do I know which ones will help me a priori?
You need the lifted representatives to factor into small primes (and a unit $\pm1$) only, which is most likely for numbers with small absolute value. Therefore you choose the representative with smallest absolute value (which is unique for odd $n$).
If that does not work, simply forget that one and look at squares of other numbers, again the representatives with smallest absolute value. Those are (roughly) similar in magnitude, about some small multiple of $\sqrt{n}$, instead of being guaranteed to be at least about $n$ if you'd stick to the old square and just choose another representative.