We can write the fractional part $\left\{{\sqrt{N^2 + k^2}}\right\} = \sqrt{N^2 + k^2} - N - n$ for some $n \ge 0$. After solving for $n$ we get $$\sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \left\{{\sqrt{N^2 + k^2}}\right\} = \sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \sqrt{N^2 + k^2} - N\, \left\lfloor{\frac{N}{2}}\right\rfloor - n_{\max} + \sum_{n = 0}^{n_{\max}} \left\lceil{\sqrt{2nN + n^2}}\right\rceil - n_{\max} \times \min \left(\left\lceil{\sqrt{2(n_{\max} + 1) N + \left(n_{\max} + 1\right)^2 }}\right\rceil - 1, \left\lfloor{\frac{N}{2}}\right\rfloor\right)$$
where $n_{\max} = \left\lfloor{\sqrt{N^2+\left\lfloor{N/2}\right\rfloor^2}}\right\rfloor-N$. See Find the asymptotic expansion as $N \rightarrow \infty$ of $\sum_{k=1}^{\left\lfloor{N/2}\right\rfloor} \left\{{\sqrt{{N}^{2}+{k}^{2}}}\right\}$ for a different form of this summation problem. I can use the Euler-Maclaurin formula for the first sum to obtain $$\sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \sqrt{N^2 + k^2} \sim \frac{1}{8} \left(\sqrt{5} + 4\operatorname{arccsch}(2)\right) N^2 + \frac{1}{4} \left(\sqrt{5} - 2\right) N + \frac{1}{12 \sqrt{5}} + (N^{-2})$$
I am looking for the asymptotic expansion of $\sum_{n = 0}^{n_{\max}} \left\lceil{\sqrt{2nN + n^2}}\right\rceil$ as $N \rightarrow \infty$. If I apply the Euler-Maclaurin formula to square root only we get $$\sum_{n = 0}^{n_{\max}} \left\lceil{\sqrt{2nN + n^2}}\right\rceil \sim \sum_{n = 1}^{n_{\max}} \sqrt{2nN + n^2} \sim \frac{1}{8} \left(\sqrt{5} - 8\operatorname{arccsch} \left(2 \sqrt{2 + \sqrt{5}}\right)\right) N^2 + \frac{1}{4}\, N - C \sqrt{N} + \frac{\sqrt{5}}{12} + \left(N^{- 1/2}\right)$$
where the constant $C$ depends on the order of approximation. $C$ approaches 0.29406 then diverges with higher orders of approximation. Numerical and other testing shows that this quadratic term is correct while the terms $N$ and $\sqrt{N}$ are not valid.
What is the better approach to approximating this sum with the ceiling function with the desired order of approximation is out to the reciprocal terms in $N$.
This is where the problem is derived: $$\sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \left\lfloor{\sqrt{{N}^{2} + {k}^{2}}}\right\rfloor = \sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \sqrt{{N}^{2} + {k}^{2}}- \sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \left\{{\sqrt{{N}^{2} + {k}^{2}}}\right\}$$
where I am looking for the expansion as $N \rightarrow \infty$ out to reciprocal terms in $N$. The problem Closed-form for Floor Sum 3 - With knowledge of inner expression is a restatement of this one. There is no solutions beyond what I already have for $\sum_{k = 1}^{\left\lfloor{N/2}\right\rfloor} \sqrt{N^2 + k^2}$.
We can write $$\sum_{n=0}^{n_{\max}} \left\lceil{\sqrt{2nN+n^2}}\right\rceil = \sum_{n=N}^{n_{\max}+N} \left\lceil{\sqrt{n^2 - N^2}}\right\rceil$$ Which is a form resembling Gauss Circle Problem.