Getting closest relative position of a point in a toroidal space

Question

I am writing a boid simulation in a toroidal space. The boids wrap around from the left to right edge and the top to bottom edge. To calculate what boids are visible, I need to find the minimum distance between those two boids' positions. I have been doing the following

Given: $$P_1 = (x_1, y_1), P_2 = (x_2, y_2)$$

1. $\Delta x = \lvert(x_2 - x_1)\rvert,\Delta y = \lvert(y_2 - y_1)\rvert$
2. Apply the toroidal wrapping on x: If $\Delta x > width/2$, $\Delta x = width - \Delta x$
3. Apply the toroidal wrapping on y: If $\Delta y > height/2$, $\Delta y = height - \Delta y$
4. Calculate the Euclidean distance: $d=\sqrt{(\Delta x)^2+(\Delta y)^2}$

How would I also get the closest relative position of $P_2$ in Euclidean space (possibly outside of the toroidal space's $width$ and $height$ restrictions) in relation to $P_1$?

I have tried to do: $$P_3 = (\Delta x + x_1, \Delta y + y_1)$$ but it doesn't work every time. For example: it worked with $P_1=(700, 520), P_2=(700, 20)$ which resulted in $P_3=(700, 560)$, but not with $P_1=(405, 20), P_2=(50, 530)$, which resulted in $P_3=(740,50)$

Here is a picture of what I am trying to find: $P_3$ being what I want to get

Answer 1

$\newcommand{\Neg}{\phantom{-}}\DeclareMathOperator{\sgn}{sgn}$Write $W$ for the width of the fundamental rectangle and $H$ for the height, and let $\sgn$ be the signum function: $$ \sgn(x) = \begin{cases} -1 & x < 0, \\ \Neg0 & x = 0, \\ \Neg1 & x > 0. \end{cases} $$

Let's say two points of the plane are avatars (of each other) if they represent the same point of the torus, i.e., their difference has the form $(mW, nH)$ for some integers $m$ and $n$.

Treating $P_{2}$ as the "origin" and assuming both $P_{1}$ and $P_{2}$ are in the same fundamental $W \times H$ rectangle, the goal is to find the nearest avatar of $P_{1}$ to $P_{2}$.

Set $m = -\sgn(x_{1} - x_{2})$ and $n = -\sgn(y_{1} - y_{2})$. Geometrically, these are "the numbers of horizontal and vertical steps," either $1$, $0$, or $-1$, to move $P_{1}$ closer to $P_{2}$.

The nearest avatar $P_{3}$ is one of at most four points: $$ P_{1},\qquad P_{1} + (mW, 0),\qquad P_{1} + (0, nH),\qquad P_{1} + (mW, nH). $$

If it matters, there is generally no continuous choice of nearest avatar as $P_{1}$ moves. That may mean your software needs to draw two or four avatars for one boid in some circumstances to avoid visual discontinuities.