Let $0<s<1$. Let $\Omega\subset\mathbb{R}^n$ be a bounded domain.
We know that $$\|(-\Delta)^{s/2}u\|_{L^2(\mathbb{R}^n)}^2=\int_{\mathbb{R}^n}\int_{\mathbb{R}^n}\frac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}dxdy$$ See e.g. Hitchhiker's guide to the fractional Sobolev spaces, page 16.
I am wondering if the equality still holds when $\mathbb{R}^n$ is replaced by $\Omega$. Namely $$\|(-\Delta)^{s/2}u\|_{L^2(\Omega)}^2=\int_{\Omega}\int_{\Omega}\frac{|u(x)-u(y)|^2}{|x-y|^{n+2s}}dxdy\ ?$$
We may assume that $u\in L^2(\mathbb{R}^n)$ and supp $u$ $\subset\Omega$.
Thanks!
Nice question, with negative answer (unless $s=0$ or $2$, of course).
The point is that the fractional Laplacian is non-local; the value of $(-\Delta )^{s/2}f $ at a point depends on $f$ at all points. In particular, a norm of $(-\Delta)^{s/2} f$ must take into account the values of $f$ everywhere; you cannot arbitrarily choose to consider only the values in $\Omega$.
There are different versions of the fractional Laplacian adapted to domains. If you take a complete orthonormal system of eigenfunctions $\phi_0, \phi_1, \phi_2\ldots$ for the Dirichlet problem on $\Omega$, you can define the so-called Dirichlet Laplacian by $$ (-\Delta)^{s}_{\mathrm{Dir}} f:=\sum_{\ell=0}^\infty \lambda_\ell^s \hat{f}(\ell) \phi_\ell, $$ where $-\Delta \phi_\ell=\lambda_\ell \phi_\ell$, and $$ \hat{f}(\ell):=\int_{\Omega} f \phi_\ell\, dx.$$ This is a version of the fractional Laplacian that is localized on $\Omega$.
I don't know what is the precise relationship between the Dirichlet Laplacian and the free-space fractional Laplacian.