let $M,n\in \mathbb{N}$ and $R=\lbrace r\in \mathbb{N} \mid \vert r- \sqrt{n}\vert <M<2\sqrt{n}\rbrace $. I have to show that the maximal value of $\vert r^2-n\vert $ for $r\in R$ is at most $(M-1)^2+2M\lceil \sqrt{n}\rceil$.
I found out that we can write each $r\in R$ as $r= \lceil \sqrt{n}\rceil + i$ with $i=-M, \dots ,M-1$. If we look at $\vert r^2-n \vert$ and consider the two cases $r^2-n\geq 0$ and $r^2-n<0$ then the first case is quite easy, but I have problems with the second case.
Then I have $r^2-n<0$ (that means $r= \lceil \sqrt{n}\rceil + i$ for $i=-M,\dots ,-1$) but if I plugin all the stuff I get $-r^2+n=n- \lceil \sqrt{n}\rceil^2 - 2i\lceil \sqrt{n}\rceil -i^2 \leq n-\lceil \sqrt{n}\rceil^2 + 2M \lceil \sqrt{n}\rceil -M^2$, but now??
Hint: You can't plug directly $i=-M$ in the last inequality, since you do not know for which $i$ the expression is maximum (note the $-M^2$...). However you have $$n-\lceil \sqrt{n} \rceil^2 -2i \lceil \sqrt{n} \rceil - i^2 \leq n-\lceil \sqrt{n} \rceil^2+ 2M\lceil \sqrt{n} \rceil -1. $$ Now note that $\lceil \sqrt{n} \rceil \geq \sqrt{n}$, hence $n- \lceil \sqrt{n} \rceil^2-1 \leq 0 \leq \cdots$