In a physical set-up, one can consider an electrostatic problem where the charge density at each point in space is a random variable, and try to find the electric potential or electric field. To be more concrete, Consider a Poisson equation $$\nabla^2\phi (\mathbf{r}) = - \rho (\mathbf{r})$$ with free boundaries where $\rho$ is given by an uncorrelated white Gaussian noise, i.e. $$\langle \rho(\mathbf{r}) \rangle = 0, \qquad\qquad\left\langle \rho(\mathbf{r}) \rho(\mathbf{r}') \right\rangle = A \delta(\mathbf{r}-\mathbf{r}').$$ If I write down solutions in the Fourier space, they look like $\mathbf{q}^{-2} \rho(\mathbf{q})$ and on average the potential is zero. However, The correlations of $\phi$ then read $$\langle \phi(\mathbf{q}) \phi(\mathbf{q}') \rangle = \frac{\langle \rho(\mathbf{q}) \rho(\mathbf{q}') \rangle}{\mathbf{q}^2 {\mathbf{q}'}^2} = \frac{A \, (2\pi)^d \delta(\mathbf{q}+\mathbf{q}')}{\mathbf{q}^4}.$$ I think this is not a well-defined expression because of $q^{-4}$ term, and I cannot make sense of it. My guess is that this is because the charge density $\rho$ can assume configurations in which it is not localized in space, and attempting to solve the Poisson equation in that scenario leads to this failure (similar to $\nabla^2 \phi = cnst.$).
On the other hand, such configurations must occur with a vanishing probability (i.e. all space being filled with charges is improbable) and so I would naively expect them not to contribute to the correlations. How can I resolve this apparent contradiction?
If I understand correctly, the steps in the calculations are as follows:
You start with the Poisson equation $$ \nabla^2\phi (\mathbf{r}) = - \rho (\mathbf{r}) \\ $$ Then take the Fourier transform of it using $f(\mathbf{q}) = \int f(\mathbf{r}) e^{-i\mathbf{q}\cdot\mathbf{r}} d\mathbf{r}$ (just using the same notation for the function and its transform; the argument separates them), getting $$ -\mathbf{q}^2\phi(\mathbf{q}) = -\rho(\mathbf{q}) \\ $$ You solve this equation getting $$ \phi(\mathbf{q}) = \frac{\rho(\mathbf{q})}{\mathbf{q}^2} \\ $$
Then you form the correlator $$ \left< \phi(\mathbf{q}) \phi(\mathbf{q}') \right> = \left< \frac{\rho(\mathbf{q})}{\mathbf{q}^2} \frac{\rho(\mathbf{q}')}{\mathbf{q}'^2} \right> = \frac{\left< \rho(\mathbf{q}) \rho(\mathbf{q}') \right>}{\mathbf{q}^2 \mathbf{q}'^2} $$ where $$ \left< \rho(\mathbf{q}) \rho(\mathbf{q}') \right> = \left< \int \rho(\mathbf{r}) e^{-i\mathbf{q}\cdot\mathbf{r}} d\mathbf{r} \int \rho(\mathbf{r'}) e^{-i\mathbf{q}'\cdot\mathbf{r}'} d\mathbf{r}' \right> = \iint \left< \rho(\mathbf{r}) \rho(\mathbf{r'}) \right> e^{-i\mathbf{q}\cdot\mathbf{r}} e^{-i\mathbf{q}'\cdot\mathbf{r}'} d\mathbf{r} \, d\mathbf{r}' \\ = \iint A \delta(\mathbf{r} - \mathbf{r'}) e^{-i\mathbf{q}\cdot\mathbf{r}} e^{-i\mathbf{q}'\cdot\mathbf{r}'} d\mathbf{r} \, d\mathbf{r}' = \int A e^{-i(\mathbf{q}+\mathbf{q}')\cdot \mathbf{r}} d\mathbf{r} = A (2\pi)^d \delta(\mathbf{q}+\mathbf{q}') $$
There is one error above. The solution to the equation $-\mathbf{q}^2\phi(\mathbf{q}) = -\rho(\mathbf{q})$ actually has a couple of extra terms: $$ \phi(\mathbf{q}) = \frac{\rho(\mathbf{q})}{\mathbf{q}^2} + a\delta(\mathbf{q}) + \mathbf{b}\cdot \nabla\delta(\mathbf{q}), \\ $$ where $a$ and $\mathbf{b}$ are constants (scalar and vector respectively).
I'm not sure that helps, however.