$$\sum_{n=1}^\infty \frac{\{H_n\}}{n^2}$$
I have managed to prove that it converges, but am having trouble with a closed form. This came from a school contest from last year, but can't really figure it out.
I came up with a numeric solution, but am having lots of trouble with the closed form. Thanks in advance.
On
Not an answer to the actual question, but some thoughts, some useful references and a derivation of a good approximation to the sum.
We can rewrite the sum as (see e.g. this answer or the answer above for the $2\zeta(3)$ evaluation) $$\sum_{n=1}^\infty \frac{\{ H_n \}}{n^2}=\sum_{n=1}^\infty \frac{H_n}{n^2}-\sum_{n=1}^\infty \frac{\lfloor H_n \rfloor}{n^2} = 2\zeta(3) - \sum_{n=1}^\infty \frac{\lfloor H_n \rfloor}{n^2}$$ and we can further rewrite $$ \begin{array}{cll}\tag{1} \sum_{n=1}^\infty \frac{\lfloor H_n \rfloor}{n^2} &=& 1\left(\frac{1}{1^2} + \ldots + \frac{1}{3^2}\right) \\&+& 2\left(\frac{1}{4^2} + \ldots + \frac{1}{10^2}\right) \\&+& 3\left(\frac{1}{11^2} + \ldots + \frac{1}{30^2}\right) \\&+& 4\left(\frac{1}{31^2} + \ldots + \frac{1}{82^2}\right) \\&+& \ldots\\ &=& \sum_{k=1}^\infty k\sum_{j=n_k}^{n_{k+1}-1}\frac{1}{j^2} \end{array} $$ where $n_k$ are defined as the smallest integer where $\lfloor H_n\rfloor = k$. Since $H_n = \log(n) + \gamma + \frac{1}{2n} + \mathcal{O}(n^{-2})$ we expect $n_k = \lfloor e^{k-\gamma} + \frac{1}{2}\rfloor$. This formula holds for $k\leq 100$ and heuristics suggest that it could hold for all $k$, see comments in OEIS A002387 and R. P. Boas, Jr. and J. W. Wrench, Jr., Partial Sums of the Harmonic Series, The American Mathematical Monthly.
Evaluating the sum requires knowledge of $n_k$ and since this is not known in the litterature (as far as I have chekced) it seems unlikely that the problem has a (known) closed form solution. Even if the simple formula for $n_k$ holds it still seems pretty hopeless to evaluate the sums, however it gives us a good starting point for an approximation. We can approximate
$$\sum_{j=n_k}^{n_{k+1}-1}\frac{1}{j^2} \approx \int_{n_k}^{n_{k+1}}\frac{{\rm d}x}{x^2} = \frac{(n_{k+1}-n_k)}{n_kn_{k+1}}$$
Now if we take $n_k = e^{k-\gamma}$ in the expression above then the integral is $(e-1)e^{\gamma-1-k}$ and the resulting sum can be evaluated analytically. A good approximation can be found by explicitly summing the first $K-1$ brackets in $(1)$ and using the approximation above to estimate the rest of the terms. This leads to
$$\sum_{n=1}^\infty \frac{\{H_n\}}{n^2} \approx 2\zeta(3)-\sum_{k=1}^{K-1} k\sum_{j=n_k}^{n_{k+1}-1}\frac{1}{j^2} -\frac{K(e-1)+1}{(e-1)e^{K-\gamma}}$$
Taking $K=5$ gives us $$2\zeta(3)-2.00822-\frac{5 (e-1) + 1}{(e-1)e^{5-\gamma}} \approx 0.32890$$
which is within $0.2\%$ of the exact result (found by summing numerically up to $n=10^7$).
Note: this is the answer to calculate $\sum_{n=1}^\infty \frac{H_n}{n^2}$ instead of the one $\sum_{n=1}^\infty \frac{\{H_n\}}{n^2}$ in the question.
First the $n$-the harmonic number can be written as $$ H_n = 1+\frac{1}{2}+\cdots+\frac{1}{n}=\int_{0}^1 \left(1+x+\cdots+x^{n-1}\right)dx =\int_0^1 \frac{1-x^n}{1-x}dx. $$ Then $$ \sum_{n=1}^\infty \frac{H_n}{n^2} =\sum_{n=1}^\infty \int_0^1 \frac{1}{n^2}\frac{1-x^n}{1-x}dx =\int_0^1 \frac{1}{1-x}\left(\sum_{n=1}^\infty\frac{1}{n^2}-\sum_{n=1}^\infty\frac{x^n}{n^2}\right)dx.$$ By the Taylor expansion of $\log(1-t)$, $$ \log(1-t) =-\left(t+\frac{t^2}{2}+\cdots+\frac{t^n}{n}+\cdots\right),$$ we can write $$ \sum_{n=1}^\infty \frac{x^2}{n^2} = -\int_0^x \frac{\log(1-t)}{t}dt $$ which is exactly the dilogarithm function $\mbox{Li}_2(x)$.
Therefore, $$\sum_{n=1}^\infty \frac{H_n}{n^2} =-\int_0^1 \frac{1}{1-x}\int_x^1 \frac{\log(1-t)}{t}dt dx =-\int_0^1 \frac{\log(1-t)}{t}\int_0^t \frac{1}{1-x}dxdt =\int_0^1 \frac{(\log(1-t))^2}{t}dt $$ The final step is to convert the above integral into some integral representation of the Riemann Zeta function. For example, using the change of variable $s=-\log(1-t)$, $$ \sum_{n=1}^\infty \frac{H_n}{n^2}=\int\frac{s^2}{e^s-1}ds=\Gamma(3)\zeta(3)=2\zeta(3). $$