I am trying to show that for nonzero integers $x, y$, the inequality $|x^2 - (1+\sqrt{2})^2y^2| \geq 1$ holds.
What I have done: I think it is enough to consider positive integers $x, y$, and let $x$ be the integer that is close enough to $(1+\sqrt{2})y$, but I don’t know how to control $|x-(1+\sqrt{2})y|$ in this situation. Can someone help me?
Not a complete answer, but hopefully this stuff will be useful for someone else to reach a complete solution. Also, the minimum I've found from some numerical tests is $x=5, y=2 \rightarrow 1.686$, so unsure if exact $1$ can be achieved.
Summary: We are done if we can prove $$\left| \lfloor x^0(y) \rceil^2 - (x^0(y))^2 \right| \geq 1 \quad \forall \; y \in \mathbb{N}, \quad x^0(y) \equiv (\sqrt{2} + 1)y$$ where $\lfloor x \rceil$ is the value of $x$ rounded to the nearest integer (not sure if there is a standard notation for this).
Explanation: One approach to solve the original question is to find the minimum value of the following, and show it's $\geq 1$ $$\min_{x,y \in \mathbb{Z} \setminus {0}} f(x,y), \quad f(x,y) \equiv |x^2 - (1+\sqrt{2})^2y^2|$$
As the author noted, $f$ is even in both $x$ and $y$; i.e. $f(x,y) = f(-x,y) = f(x,-y)$. So we can simplify the domain to just natural numbers, $$\min_{x,y \in \mathbb{N}} f(x,y)$$
Now suppose we relax the integrality requirement on $x$. Then the minimum value is $f(x^0(y),y) = 0$ for any $y \in \mathbb{N}$, where $x^0(y) \equiv (\sqrt{2} + 1)y$. Clearly, $x^0(y)$ will always be irrational for any natural number $y$. So this can never be the solution to the above minimization problem. But if we analyze $f(x,y)$ for a given $y$, we can observe that the value will increase as $x$ moves away from $x^0(y)$. So for a given $y$, the minimum value of $f(x,y)$ when $x$ is also a natural number is the closest integer to $x^0(y)$, which I'm denoting by $\lfloor x^0(y) \rceil$.
Then we don't need to check all pairs of $x$ and $y$; just $\lfloor x^0(y) \rceil$ for every $y$. Then the minimization problem becomes single variable, $$\min_{y \in \mathbb{N}} f(\lfloor x^0(y) \rceil,y)$$
So one could iterate through all the natural numbers, calculating the value for each and recording the minimum so far. I ran up to $10^6$, and got $y = 2, x^0(y) =4.83, f(\lfloor x^0(y) \rceil,y) = 1.686$.
But let's look at the function itself. $$f(\lfloor x^0(y) \rceil,y) = |(\lfloor x^0(y) \rceil)^2 - (1+\sqrt{2})^2y^2| = |(\lfloor x^0(y) \rceil)^2 - (x^0(y))^2 |$$
It's the difference between the square of $x^0(y)$ and the square of the rounded value of $x^0(y)$, and so proportional to the gap between $x^0(y)$ and the nearest integer to it. Let this gap be $\epsilon(y)$, where $\lfloor x^0(y) \rceil = x^0(y) + \epsilon(y)$ and so $-1 \leq \epsilon(y) \leq 1$. Then we get, since $x^0(y) \geq 2$, $$f(\lfloor x^0(y) \rceil,y) = |\epsilon(y)|\left(2x^0(y) +\epsilon(y)\right)$$