Suppose $X$ and $Y$ each are subsets of $\mathbb{R}$ that are bounded below and unbounded above (and therefore infinite).
Given $\varepsilon>0,\ $ do there exist $\ x_1,\ x_2,\ x_3 \in X;\ x_1 < x_2 < x_3;\ \quad y_1,\ y_2,\ y_3 \in Y;\ y_1 < y_2 < y_3,\ $ such that
$$ 1 - \varepsilon < \left\lvert \frac{ \frac{x_2 - x_1}{x_3 - x_1}}{\frac{y_2 - y_1}{y_3 - y_1}} \right\rvert < 1+\varepsilon\quad ?$$
This is equivalent to saying that we can find (many) pairs of triplets $(x_1,x_2,x_3)$ and $(y_1,y_2,y_3),$ such that $\ \frac{x_2 - x_1}{x_3 - x_1}$ and $ \frac{y_2 - y_1}{y_3 - y_1}\ $ are relatively arbitrarily close, which means that each pair of triplets $(x_1,x_2,x_3)$ and $(y_1,y_2,y_3),$ "look similar to each other" albeit for a scaling and translation.
Is $X = \{2^n:\ n\in\mathbb{N} \};\ Y = \{3^n:\ n\in\mathbb{N} \}\ $ a counter-example? I am not sure...
And maybe you construct a back-and-forth counter-example if $X$ and $Y$ are finite sets of arbitrary size, but I'm not sure this construction would extend to $X$ and $Y$ being infinite sets. On the other hand, it seems plausible that there are not good back-and-forth counter-examples if $X$ and $Y$ are finite sets of arbitrary size by some application of the Pigeonhole principle. But I'm not sure about what's true, so it will require more thought.
Also, Stolz-Cesàro theorem might play a role.
Let $C>1$ be a constant and $(a_n)$ be any sequence s.t. $a_n\ge a_{n-1}+1$ and $a_{n}\ge Ca_{n-1}^2+a_{n-2}$.
Let $i> j> k$ and $r> s> t$ be indices such that $i > r$.
$x = \dfrac{a_j-a_k}{a_i-a_k}$ is at most $\dfrac{a_{i-1}}{a_i-a_{i-2}}$ since $a_j-a_k\le a_j\le a_{i-1}$ and $a_i-a_k\ge a_i-a_{i-2}$.
$y = \dfrac{a_s-a_t}{a_r-a_t}$ is at least $\dfrac{1}{a_{i-1}}$ since $a_s-a_t\ge a_s-a_{s-1}\ge 1$ and $a_r-a_t\le a_r\le a_{i-1}$.
Now notice that $a_i\ge Ca_{i-1}^2+a_{i-2}\implies \dfrac{1}{a_{i-1}}\ge C\dfrac{a_{i-1}}{a_i-a_{i-2}}$, therefore, $y\ge Cx$.
This leads to the conclusion that, for any strictly decreasing triplets of indices $(i, j, k)$ and $(r, s, t)$ with $i\ne r$, one should have $\dfrac{\dfrac{a_j-a_k}{a_i-a_k}}{\dfrac{a_s-a_t}{a_r-a_t}} > C$ or $\dfrac{\dfrac{a_j-a_k}{a_i-a_k}}{\dfrac{a_s-a_t}{a_r-a_t}} < \dfrac{1}{C}$.
Now just take $X$ and $Y$ to be disjoint subsequences of $(a_n)$! :)
For example, $a_n = 3^{3^n}$ is such a sequence for $C = 2$, so you could take $X = \{3^{3^{2n+1}}:n\in\mathbb N\}$ and $Y = \{3^{3^{2n}}:n\in\mathbb N\}$. That is similar to what was been discussed in the comments.
Extra: I was curious to know if $X = \{2^n: n\in\mathbb N\}$ and $Y = \{3^n: n\in\mathbb N\}$ was a (much simpler) counter-example to your question, so I made this post. As it turns out, it is not.