I came across with that notation in one book. They defined
$$\overline{X_n}=\frac{X_1+...+X_n}{n}.$$
Then they define $n\in \mathbb N$, $k_n\in\mathbb Z$ such that:
$$[k_n]^2\leq n\leq [k_n+1]^2.$$
Now they say:
$$\overline{X_n}-\frac{[k_n]^2}{n}\overline{X_{k_n^2}}=\frac{1}{n}\sum\limits_{j=k^2_n+1}^nX_j.$$
However the left-hand side of the equation is $$\frac{X_1+X_2+...+X_n}{n}-\frac{k^2_n}{n}\left(\frac{X_1+X_2+...+X_{k^2_n}}{k^2_n}\right)=\frac{-X_{n+1}-X_{n+2}-...-X_{k^2_n}}{n}$$ $$\stackrel{?}{=}\frac{1}{n}\sum\limits_{j=k^2_n+1}^nX_j.$$
Any help for this notation would be appreciated, I have never seen this before. Also the sum looks like shifted because the real result of $\overline{X_n}-\frac{[k_n]^2}{n}\overline{X_{k_n^2}}$ starts with $-X_{n+1}$ and ends with $-X_{k^2_n}.$
Consider that you have $k_n^2 \leq n$ so the only members of the sum left are the ones that go from $k_n^2+1$ to $n$ ${}{}$