This a question from a statistical spatial analysis.
Let $\Theta = (\beta, \sigma^2, \tau^2, \phi)$. Assume that $Y | \Theta \sim \mathcal{N}(X\beta, \sigma^2R(\phi) + \tau^2I).$ Suppose that $\phi$ is fixed and that the prior distribution is $p(\beta, \sigma^2, \tau^2) \propto (\sigma^2\tau^2)^{-1}$, a quite standard non-informative prior often chosen in non-spatial analysis contexts. Show that the posterior distribution $p(\beta, \sigma^2, \tau^2 | y)$ is improper.
My approach began with attempting to solve for $\beta$ first by identifying a normal distribution. Then, I thought $\sigma^2$ and $\tau^2$ would be more tractable. However, I am unsure if straightforward integration is the correct method.
Let $\Sigma = \sigma^2R(\phi) + \tau^2I$
then
$$p(\beta, \sigma^2, \tau^2 | y) \propto \int_{\tau^2} \int_{\sigma^2} \int_{\beta} \frac{1}{\tau^2} \frac{1}{\sigma^2} \cdot \frac{1}{(2\pi)^{n/2}} |\Sigma|^{-1/2} \exp \left\{ -\frac{1}{2} (Y - X\beta)^T \Sigma^{-1} (Y - X\beta) \right\}$$
Define
$$ XA=Y $$
$$ X^TXA=X^TY $$
$$ A=(X^TX)^{-1}X^TY $$
We can write $$\propto \int_{\tau^2} \int_{\sigma^2} \int_{\beta} \frac{1}{\tau^2} \frac{1}{\sigma^2} \cdot \frac{1}{(2\pi)^{n/2}} |\Sigma|^{-1/2} \exp \left\{ -\frac{1}{2} \left(\beta - (X^TX)^{-1}X^TY \right)^T X^T\Sigma^{-1} X \left( \beta - (X^TX)^{-1}X^TY \right) \right\} $$ Define $$ \Sigma_\beta = X^T\Sigma^-1X $$
$$ \propto \int_{\tau^2} \int_{\sigma^2} \int_{\beta} \frac{1}{\tau^2} \frac{1}{\sigma^2} \cdot \frac{1}{(2\pi)^{n/2}} |\Sigma|^{-1/2} \dfrac{|\Sigma_\beta|^{-1/2}}{|\Sigma_\beta|^{-1/2}} \exp \left\{ -\frac{1}{2} \left(\beta - (X^TX)^{-1}X^TY \right)^T\Sigma^{-1}_{\beta} \left( \beta - (X^TX)^{-1}X^TY \right) \right\} $$
$$\propto \int_{\tau^2} \int_{\sigma^2} \int_{\beta} \frac{1}{\tau^2} \frac{1}{\sigma^2} \cdot \frac{1}{(2\pi)^{n/2}} |\Sigma|^{-1/2} \exp \left\{ -\frac{1}{2} (Y^T\Sigma^{-1} Y- Y^T\Sigma^{-1}X\beta-\beta^TX^T\Sigma^{-1} Y+\beta^TX^T\Sigma^{-1}X\beta) \right\}$$
Then integrating respect $\beta$, is just a normal distribution in all parametric space so is just one.
$$ = \int_{\tau^2} \int_{\sigma^2} \frac{1}{\tau^2} \frac{1}{\sigma^2}\dfrac{1}{2\pi^{(n/2-p/2)}}|\Sigma|^{-1/2}|\Sigma_\beta|^{1/2} $$
after this i tend to think is just is a straightfoward integration but I'm not sure. I would greatly appreciate any advice or alternate perspectives that could simplify or clarify this problem.