Suppose that $a, b$ are postive numbers and $a + b \in \mathbb{Z} {}_+$. Find the maximal constant $c$, s.t. $$ \{ a^2 \} + \{ b^2 \} \leqslant 2 - \frac{c}{(a + b)^2} $$ for all $a, b$. Here $\{ x \}$ is the fractional part of $x$.
It is said the $c=\dfrac{3}{4}$ is best, but I don't see why.
On
A complete proof.
Let $n=a+b\in\Bbb N\setminus\{0\}$ and $0<a<b$. Then we can write \begin{align} &a_n(x)=\frac n2-x& &b_n(x)=\frac n2+x& \end{align} with $0\leq x<\frac n2$. For each $n$ define the function \begin{align} &f_n:\left[0,\frac n2\right)\to[0,2]& &x\mapsto\{a_n(x)^2\}+\{b_n(x)^2\} \end{align} Then we are looking for $$c=\inf_{n>0}\inf_{0\leq x<n/2}(2-f_n(x))n^2=\inf_{n>0}n^2\left(2-\sup_{0\leq x<n/2}f_n(x)\right)$$
Let $S_n$ be the set of $x_0\in\left[0,\frac n2\right)$ such that $\{a_n(x_0)^2\}=0$ or $\{b_n(x_0)^2\}=0$. Then $S_n$ has finitely many elements and $f_n$ has positive derivative at each point $x\notin S_n$. Consequently, $$\sup_{0\leq x<n/2}f_n=\sup_{x_0\in S_n}\lim_{x\to x_0^-}f_n(x)$$
We have $\{a_n(x_0)^2\}=0$ if and only if $a_n(x_0)^2\in\Bbb N$, hence $x_0=n/2-\sqrt u$ for some $\Bbb N\ni u\leq (n/2)^2$, hence $$\lim_{x\to x_0^-}f_n(x)=1+\{(n-\sqrt u)^2\}= \begin{cases} 2-\{2n\sqrt u\}&\sqrt u\notin\Bbb N\\ 0&\sqrt u\in\Bbb N \end{cases}$$ Similarly, $\{b_n(x_0)^2\}=0$ if and only if $b_n(x_0)^2\in\Bbb N$, hence $x_0=\sqrt u-n/2$ for some $u\in\Bbb N$ satisfying $(n/2)^2\leq u\leq n^2$, hence $$\lim_{x\to x_0^-}f_n(x)=\{(\sqrt u-n)^2\}+1= \begin{cases} 2-\{2n\sqrt u\}&\sqrt u\notin\Bbb N\\ 0&\sqrt u\in\Bbb N \end{cases}$$
Consequently, we get $$c=\inf_{n>0}\inf_{0<u<n^2\\\sqrt u\notin\Bbb N}n^2\{2n\sqrt u\}$$
Write $4n^2u=q^2+r$ with $0\leq r\leq 2q$. Note that $\sqrt u\notin\Bbb N$ implies $r\neq 0$, and $q^2+r\equiv 0\pmod 4$ implies $r\neq 1$ and $r\neq 2$, so that $r\geq 3$. Then we have \begin{align} n^2\{2n\sqrt u\} &=n^2\left(\sqrt{q^2+r}-q\right)\\ &=\frac{n^2r}{\sqrt{q^2+r}+q}\\ &\geq\frac{n^2r}{2\sqrt{q^2+r}}\\ &=\frac{nr}{4\sqrt u}\\ &\geq\frac 34 \end{align} which proves $c\geq\frac 34$.
Finally, the maximality of $c=\frac 34$ follows by considering \begin{align} &n=12m^2+1& &u=\left(6m^2+\frac{21m-5}4\right)^2+3\left(6m^2-\frac{11m+1}4\right)^2& &q=288m^4-36m^3+48m^2-9m+2 \end{align} for $m>0$ and $m\equiv 1\pmod 4$. Then $4n^2u=q^2+3$ and $n^2\{2n\sqrt u\}=n^2\left(\sqrt{q^2+r}-q\right)\to (3/4)^+$ as $m\to+\infty$.
I am trying to make this intuitive and narrative, and you can work out yourself to make it rigorous.
Let us see why $3/4$ should be the optimal answer. Intuitively, if we are looking for a pair $(a, b)$ such that the corresponding $c$ is large, then $a^2, b^2$ should be slightly smaller and extremely close to an integer. Let us say $a = \sqrt u - \epsilon_1, b = \sqrt v - \epsilon_2$ where $u, v\in \mathbb Z$. Now as we are expecting square roots playing a central role, let us recall the following property:
If $\epsilon$ is close to zero, $(\alpha \pm \epsilon)^2 \approx \alpha^2 \pm 2\alpha\epsilon$.
Without loss of generality let $a \leq b$. As we are requesting $a+b$ to be close to an integer, we write $\sqrt u + \sqrt v = n+\epsilon$ for an integer $n$, and keep in mind that $\epsilon$ is slight larger than $0$. If we have determined $u, v$ and $n$, how do we interpret $\epsilon$ in terms of $u, v$ and $n$? Well, let’s try to eliminate the square roots via squaring:
$$ (\sqrt u + \sqrt v)^2 - n^2 \approx 2n\epsilon; $$
on the other hand,
$$ (\sqrt u + \sqrt v)^2 - n^2 = 2\sqrt{uv} - (n^2 - u- v). $$
As we want to minimize $\epsilon$, the naive idea is to let $2\sqrt{uv}$ get close to $(n^2 - u - v)$, equivalently, $uv$ gets close to $(n^2 - u - v)^2 /4$. If $uv$ is a square, which is a special case happening when $\sqrt u = m\sqrt v$, one needs to discuss and rule out that case. For the general case, if $uv$ is not a square, the smallest difference is
$$ uv - \frac{(n^2 - u - v)^2}{4} \geq \frac 34. $$
Therefore, approximately $$ 2\sqrt{uv} - (n^2 - u - v) \geq \frac 32 \frac 1{2\sqrt{uv}} = \frac 3{4\sqrt{uv}}. $$
Hence, approximately $$ \epsilon \geq \frac 3{8n\sqrt{uv}}. $$
Now, suppose that $u, v, n$ and $\epsilon$ are fixed, and we want to tweak $a, b$ so that $\{a^2\} + \{b^2\}$ attains its maximum value, equivalently, $a^2+ b^2$ is maximized. By some observations it happens when $b - a$ is maximized. Conceptually, when $b \to \sqrt v$ and $a \to \sqrt u - \epsilon = n - \sqrt v$. The displacement between 2 and $\{a^2\} + \{b^2\}$ will then be the displacement between $a^2$ and $\sqrt u^2$, which is approximately greater than
$$ 2\epsilon \sqrt u = \frac 3{4n\sqrt v}. $$
If we fix $u$ and take $v \to \infty$, then $\sqrt v/n \to 1$, so we get
$$ 2 - (\{a^2\} + \{b^2\}) \geq \frac 3{4n^2}. $$
The next question we need to ask is, is $3/4$ really the optimal constant? We can see that, all of our optimization equivalently relies on the fact that $v$ is large. So we shall construct the triple $(u, v, n)$ for arbitrarily large $v$. This is somewhat too hard; maybe we should first fix a small $u$.
The inequality which first involved $3/4$ is an equality only when $uv = ((2x+1)/2)^2 + 3/4$ for some integer $x$, that is, when $uv = x^2 + x + 1$. This is odd, so we need to pick an odd $u$. Let we pick $u=3$.
The reformulation of our question will then be, how do we pick a $\sqrt v$ which is just slightly larger than some $n - \sqrt3$? Equivalently we want some $v$ which is slightly larger than some $n^2 + 3 - 2n\sqrt3$. We more or less need an even number $e$ such that $e\sqrt3$ is slightly larger than an integer. This suggests us to solve a Pell equation
$$ f^2 - 3e^2 = o, $$
where $o$ is a small negative integer. By taking congruent modulo 4, the smallest possible is $o =-3$.
The general solutions satisfies $e\sqrt3 - f = \sqrt3(\sqrt3 - 2)^{2m+1}$. And indeed plugging all the way back you will get values $v$ which actually works (this is not hard to check by plugging in the relations)! The three smallest $v$’s are 127 (indeed this is the example provided by Peter Košinár in the comment, as $24218/2149$ is the seventh term continued fraction of $\sqrt{127}$), 32137, and 6346711.
An interesting question will be to find other families of pairs $(u, v)$ such that the constant $3/4$ will be approached. Somehow by intuition it seems that all odd primes $u$ with $u \ | \ (x^2 + x + 1)$ for some $x$ (or equivalently $(\frac{-3}{u}) \neq -1$) will have a chance, but I can not tell if this is true.