For $0< \alpha< 2$ the fractional Laplacian of order $\alpha$ is defined up to multiplicative constants by \begin{equation*} (-\Delta)^{\alpha/2}u(x) = \int_{\mathbb R^n}\frac{u(x) - u(y)}{|x - y|^{n + \alpha}}\; dy, \end{equation*} where the integral is understood in the principal value sense and, for my purposes, $u:\mathbb R^n \to \mathbb R$ is as smooth and rapidly decaying as you want. For such $u$, define the Kelvin transform of $u$ to be s\begin{equation*} v(x) = \left(\frac 1{|x|}\right)^{n + \alpha}u(\bar x), \end{equation*} where $\bar x = \frac{x}{|x|^2}$.
My question is about the relationship between the fractional Laplacian $(-\Delta)^{\alpha/2}$ and the Kelvin transform. Is there a direct proof for the equality \begin{equation*} \left((-\Delta)^{\alpha/2}v\right)(x) = \left(\frac{1}{|x|}\right)^{n + \alpha}\left((-\Delta)^{\alpha/2}u\right)(\bar x)? \end{equation*} where by ``direct'' I mean only using the above definition of $(-\Delta)^{\alpha/2}$ and possibly a slick change of variable? \
Here is an attempt that is not quite giving me what I want but will hopefully provide a starting point for suggestions. Below I use the change of variable $y = \bar z = \frac z{|z|^2}$, $dy = \frac{1}{|z|^{2n}}dz$ together with the equality \begin{equation*} |\bar x - \bar z| = \frac{|x - z|}{|x||z|}. \end{equation*} \begin{eqnarray*} \left((-\Delta)^{\alpha/2}u\right)(\bar x) & = & \int_{\mathbb R^n}\frac{u(\bar x) - u(y)}{|\bar x - y|^{n + \alpha}}dy \\ & = & \int_{\mathbb R^n}\frac{u(\bar x) - u(\bar z)}{|\bar x - \bar z|^{n + \alpha}}\frac 1{|z|^{2n}}dz \\ & = & \int_{\mathbb R^n}\frac{u(\bar x) - u(\bar z)}{|x - z|^{n + \alpha}}\frac{(|x||z|)^{n +\alpha}}{|z|^{2n}}dz \\ & = & |x|^{n + \alpha} \int_{\mathbb R^n} \frac{\left(\frac{1}{|z|}\right)^{n - \alpha}u(\bar x) - \left(\frac 1{|z|}\right)^{n - \alpha} u(\bar z)} {|x - z|^{n+ \alpha}}dz \\ & = & |x|^{n + \alpha} \int_{\mathbb R^n} \frac{\left(\frac{|x|}{|z|}\right)^{n - \alpha}v(x) - v(z)}{|x - z|^{n+ \alpha}}dz. \end{eqnarray*}
Edit:
As it turns out the above computation is just one step away from the desired equality. The function $\Phi(x) = |x|^{n - \alpha}$ is the fundamental solution for $(-\Delta)^{\alpha/2}$ in the sense that (still up to a multiplicative constant)
\begin{equation*}
(-\Delta)^{\alpha/2}\Phi = \delta_0.
\end{equation*}
Therefore, continuing the above computation for $x\neq 0$ gives
\begin{eqnarray*}
\left((-\Delta)^{\alpha/2}u\right)(\bar x)
& = &
|x|^{n + \alpha}\int_{\mathbb R^n} \frac{\left(\frac{|x|}{|z|}\right)^{n - \alpha}v(x) - v(z)}{|x - z|^{n+ \alpha}}dz
\\
& = &
-v(x)|x|^{2n}\int_{\mathbb R^n} \frac{|x|^{\alpha - n} - |z|^{\alpha -n}}{|x - z|^{n +\alpha}}dz
\\
&&
+
|x|^{n + \alpha}\int_{\mathbb R^n}\frac{v(x) - v(z)}{|x - z|^{n + \alpha}}dz
\\
& = &
-v(x)|x|^{2n}\left((-\Delta)^{\alpha/2}\Phi\right)(x) + |x|^{n + \alpha}\left((-\Delta)^{\alpha/2}v\right)(x)
\\
& = &
|x|^{n + \alpha}\left((-\Delta)^{\alpha/2}v\right)(x),
\end{eqnarray*}
which is the desired equality. However, the only proof that I know for the fact that $(-\Delta)^{\alpha/2}\Phi = \delta_0$ relies on the Fourier transform of $\Phi$. Thus, this entire computation is still not as ``direct'' as I would like, but I suppose it will do for now.
SUMMARY.
I am going to prove that $$\tag 1 (-\Delta)^{\frac\alpha 2}_x u = \lvert \bar x \rvert^{-\alpha-d}(-\Delta)_{\bar x}^\frac\alpha 2\bar u,$$ where $\bar x = \frac x {\lvert x \rvert^2}$ and $\bar u(\bar x)=\lvert \bar x \rvert^{\alpha -d}u(x).$
INTRODUCTION.
I have noticed quite a few times that the fractional Laplacian with positive exponent $(-\Delta)^{\alpha/2}$ is much messier than its inverse $$ (-\Delta)^{-\alpha/2} v(x)=C\int_{\mathbb R^d} \frac{ v(y) }{ \lvert x- y \rvert^{d-\alpha}}\,dy.$$ So in this answer I am going to work with the latter, which is also known as Riesz potential; see the paper of Kwaśnicki 10 definitions of fractional Laplacian, Theorem 1.1 (i). There, you will also find the exact value of the constant $C>0$, which depends on $d$ and on $\alpha$. After the derivation of the correct transformation formula for $(-\Delta)^{-\alpha /2}$, taking inverses will yield (1).
The Riesz potential is a proper convolution, with a locally integrable kernel. On the other hand, $(-\Delta)^{\alpha/2}$ is given by a crazy singular integral. This is the reason I give to myself to justify why $(-\Delta)^{-\alpha/2}$ is better behaved than $(-\Delta)^{\alpha/2}$.
PROOF.
We apply the change of variable $$ \bar x=\frac x {\lvert x\rvert^2}, \quad \bar y=\frac y {\lvert y\rvert^2},\quad v(x)=\bar v (\bar x) W(\bar x), $$ where the weight $W$ is to be determined. Using the Jacobian formula $dy=\lvert \bar y \rvert^{-2d}d\bar y$ and the "chordal distance formula" (terminology mutuated from the stereographic projection) $$ \lvert \bar x - \bar y \rvert = \frac{ \lvert x - y \rvert}{\lvert x\rvert \lvert y\rvert}, $$ we obtain $$ (-\Delta)^{-\alpha/2}v(x)= C\int_{\mathbb R^d} \frac{W(\bar y)\bar v (\bar y) \lvert \bar x\rvert^{d-\alpha}}{\lvert \bar y\rvert^{d+\alpha} \lvert \bar x- \bar y\rvert^{d-\alpha}}\, d\bar y, $$ so letting $$W(\bar y)=\lvert \bar y\rvert^{d+\alpha}$$ yields $$\tag 2 (-\Delta)^{-\alpha/2}_xv=\lvert \bar x \rvert^{d-\alpha} (-\Delta)^{-\alpha/2}_{\bar x}(\lvert \bar x\rvert^{d+\alpha}v) .$$
Now we show that (2) implies (1). We let $u=(-\Delta)^{-\alpha/2}v$, so that $v=(-\Delta)^{\alpha/2}u$ and so (2) reads $$ u=\lvert \bar x\rvert^{d-\alpha}(-\Delta)^{-\alpha/2}_{\bar x}\left( \lvert \bar x\rvert^{d+\alpha} (-\Delta)^{\alpha/2}_x u\right), $$ from which we conclude $$ (-\Delta)^{\alpha/2}_{\bar x} \left( \lvert \bar x \rvert^{\alpha - d} u\right) = \lvert \bar x \rvert^{d+\alpha} (-\Delta)^{\alpha/2}_x u, $$ which is (1). The proof is complete.