I have seem the following definition
$$D^{1,2}(\mathbb{R}^N) := \{u \in L^{2^*}(\mathbb{R}^N) : |\nabla u| \in L^2(\mathbb{R}^N)\}.$$
I would like a reference that deals with this space deeply. I know this space is related with the best Sobolev constant. Specifically I'd like to know:
a) How this space was born?
b) This space has a norm or an inner product?
I know this space is nonempty, because from Brezis (Functional Analysis, Theorem 9.9) that exists a constant $c > 0$ such that $$ \qquad |u|_{L^{2^*}(\mathbb{R}^N)} \leq c \, \|\nabla u \|_{L^{2}(\mathbb{R}^N)},\; \forall u \in W^{1,2}(\mathbb{R}^N). $$ So, $W^{1,2}(\mathbb{R}^N) \subset D^{1,2}(\mathbb{R}^N)$ (is it right?)
I need to understand this space very well and so far I haven't found any reference with this. Any help or reference is very welcome.
Yes, $W^{1,2} \subset D^{1,2}$, and the difference of the two spaces is in the decay at infinity.
The first remark is that the two spaces are locally similar. If $u\in D^{1,2}$, then it is in $L^{2^*}$ and so in $L^2_{\text{loc}}$ (that is in $L^2(\Omega)$ for any compact $\Omega$). So the difference is at infinity (i.e. outside compact sets).
If you look at the behavior at infinity, the idea is that functions in $W^{1,2}$ have to be in $L^2$, which can be seen as a "faster decay" than $L^{2^*}$. For example, if you take locally nice radially decreasing functions such as $f(x) = \langle x\rangle^{-n}$ (where I wrote $\langle x\rangle = (1+|x|^2)^{1/2}$), then $f\in L^p(\Bbb R^N)$ if and only if $n > N/p$. So for example, if $N=3$ then $2^* = 6$, and $\langle x\rangle^{-1} \in D^{1,2}$ but $\langle x\rangle^{-1} \notin W^{1,2}$.
Now the crucial point is that the $L^2$ decay is not necessary to prove Sobolev inequalities. All what you need is $\nabla u \in L^2$ and $u$ "decays at infinity", because you still want to remove constants functions. But of course any space where Sobolev inequalities hold will be included in $D^{1,2}$. Hence, in some sense, $D^{1,2}$ is the largest space where Sobolev inequalities hold.
If you look for example in the book Analysis by Lieb and Loss (Sections 8.2 and 8.3) you will see that they actually initially define another space $$ D^1 = \{u\in L^1_{\text{loc}}\mid \nabla u\in L^2, u \text{ vanishes at infinity}\} $$ where $u$ "vanishes at infinity" means that for any $\lambda>0$, $\{x\mid u(x)>\lambda\}$ has finite measure. They then prove that the Sobolev inequality holds in $D^1$. Hence, $D^1 \subseteq D^{1,2}$. On the other hand, if $u\in D^{1,2}$, then it is in $L^{2^*}$, so in particular in $L^1_{\text{loc}}$. Moreover, if $\{x\mid u(x)>\lambda\}$ has infinite measure, then $$ \int_{\Bbb R^N} |u|^{2^*} \geq \int_{\{x\mid u(x)>\lambda\}} |u|^{2^*} \geq \int_{\{x\mid u(x)>\lambda\}} |\lambda|^{2^*} = \infty. $$ Hence $u\in D^1$, and so we conclude that $$ D^1 = D^{1,2}. $$ That is, $D^{1,2}$ is really just the space of functions decaying at infinity and such that $\nabla u\in L^2$.
This space is also sometimes called an homogeneous Sobolev space, and denoted by $\dot{H}^1$ and can also be defined as the completion of $C^\infty_c$ with respect to the $D^{1,2}$ norm, or by taking the quotient space so as to identify functions up to an addition of a constant function.
You can put the inner product $$ \langle u,v\rangle = \int \nabla u\cdot\nabla v. $$