Fourier transform of $$ -\frac{1}{2}\iint K(|\mathbf{r}-\mathbf{r}'|)n_1(\mathbf{r})n_1(\mathbf{r}')\mathrm{d}\mathbf{r}\mathrm{d}\mathbf{r}' $$ is $$ \frac{1}{2}\frac{1}{(2\pi)^3}\int K(\mathbf{q})|n(\mathbf{q})|^2\mathrm{d}\mathbf{q} $$
where $$ n(\mathbf{q})=\int n_1(\mathbf{r})\exp(-i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{r} $$ $$ n_1(\mathbf{r})=\frac{1}{(2\pi)^3}\int n(\mathbf{q})\exp(i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{q} $$
It seems like to use the Fourier transform of the convolution... And not sure how to deal with the absolute in function K.
Suddenly, almost solved and understood.
Step 1: Insert $$ n_1(\mathbf{q})=\frac{1}{(2\pi)^3}\int n(\mathbf{q})\exp(-i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{q} $$ into the first Eq.
Step 2: Change the variable $$ \mathbf{q}\rightarrow-\mathbf{q} $$
Then
$$ -\frac{1}{2}\iint K(|\mathbf{r}-\mathbf{r}'|)n_1(\mathbf{r}')\frac{1}{(2\pi)^3}\int n(\mathbf{q})\exp(i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{q}\mathrm{d}\mathbf{r}\mathrm{d}\mathbf{r}' $$
$$ =\frac{1}{2}\frac{1}{(2\pi)^3}\iint K(|\mathbf{r}-\mathbf{r}'|)n_1(\mathbf{r}')\int n(\mathbf{q})\exp(-i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{q}\mathrm{d}\mathbf{r}\mathrm{d}\mathbf{r}' $$
$$ =\frac{1}{2}\frac{1}{(2\pi)^3}\int \left[\iint K(|\mathbf{r}-\mathbf{r}'|)n_1(\mathbf{r}')\mathrm{d}\mathbf{r}'\exp(-i\mathbf{q}\cdot\mathbf{r})\mathrm{d}\mathbf{r}\right]n(\mathbf{q})\mathrm{d}\mathbf{q} $$
$$ =\frac{1}{2}\frac{1}{(2\pi)^3}\int \left[K(\mathbf{q})n(\mathbf{q})\right]n(\mathbf{q})\mathrm{d}\mathbf{q} $$
$$ =\frac{1}{2}\frac{1}{(2\pi)^3}\int K(\mathbf{q})|n(\mathbf{q})|^2\mathrm{d}\mathbf{q} $$
But, why the absolute can be ignored?