I am interested in getting to know for which values of $\gamma >0$ we have that $$ \lim_{n \to \infty} \frac{1}{n^2}\sum_{k=1}^n k^\gamma <\infty $$
Or more specifically, for which values $\gamma$ does this limit vanishes. A very bad bound gives us \begin{align} \frac{1}{n^2}\sum_{k=1}^n k^\gamma &\le \frac{1}{n^2} \sum_{k=1}^n n^\gamma \\&\le n^{1+\gamma-2} \longrightarrow 0 \end{align} if $\gamma < 1 $. It is very likely that you can still have the same result for larger values, as in the bound used is very really bad to bound the first values.
Would anyone have a tip or maybe a reference?
You're almost done. The sum is increasing in $\gamma$, so all you need to do is show that the limit is greater than zero when $\gamma=1$.
You can do this computation directly.