The following derivation cames from calculations related to the Laplace equation and its fundamental solution. Let $g(x)$ be a test-function (meaning compact support and infinitely differentiable), and we are on the space $\mathbb R^n$, then $$ \int_{\partial B(0,\delta)} \frac{1}{n\alpha(n)|x|^{n-1}} g(x) dS(x) = \frac{1}{n\alpha(n)\delta^{n-1}} \int_{\partial B(0,\delta)} g(x) dS(x) $$ where $\alpha(n)$ denotes volume of the unit all in $\mathbb R^n$. The derivation comes form this document, page 5, second step.
Why does this hold? For example let $n=2$, if I compute for $\delta > 0$ $$ \int_{\partial B(0,\delta)} \frac{1}{|x|} dx = \int_0^{2\pi} \int_0^{\delta} \frac{1}{r} r dr d\theta = 2\pi \int_0^{\delta} 1 dr = 2\pi \delta $$ which is not equal to $1/\delta$?
In your example, you integrate over $B(0,\delta)$, the ball of radius $\delta$. Note that on the sphere $\partial B(0,\delta) = \{x \in \mathbb R^n \mid |x| = \delta\}$ we have $|x| = \delta$ by the very definition of the set. That is $\frac 1{|x|}$ is constant on the sphere, as is every power of it.
For $n = 2$, $B(0,\delta) = \delta S^1$, which has length $2\pi \delta$, giving $$ \int_{\delta S^1} \frac 1{|x|} \, dx = \int_{\delta S^1} \frac 1\delta \, dx = 2\pi\delta \cdot \frac 1{\delta} = 2\pi $$ Note that for $g=1$, $\int_{\partial B(0,\delta)}\,g(x)\, dS(x) = n\alpha(n)\delta^{n-1}$ cancels the denominator.