Improper Uniform Prior Distribution

Question

In Bayesian method, choosing the prior distribution is an important step when using the Bayesian method. When choosing prior, we consider the prior knowledge to choose which prior distribution is the best for our problem. By hold to Laplace postulate 200 years ago "When nothing is known about X in advance, let the prior p(x) be a uniform distribution, that is, let all possible outcomes of X have the same probability." So when we have no knowledge about the parameter that we want to estimate, we use uniform distribution as the prior distribution. But, when the parameter space is infinite, that p.d.f prior is improper, which means, does not integrate/sum to one. Can this kind of prior (re: improper uniform prior) be used, though it's not satisfying p.d.f properties that the integral/sum over the parameter space is not 1? Does anyone have a link where I can get the proper definition or anything about the improper prior?

Answer 1

Yes, you can use uniform priors even if they are improper, but it might not always be wise to do so. For example, you will perhaps encounter the "uniform" prior for the variance in a normal distribution, where it is specified as $$ p(\sigma^2)\propto 1 $$ which essentially spreads the density over the entire positive real line. Naturally, it doesn't integrate to $1$ and is improper.

Edit: Usually, if you're interested in estimating parameters, you will be fine, because the posterior distribution is well-defined and integrates to 1. However, it complicates model comparisons. For instance, to compute the Bayes factor, you would need the marginal likelihood (i.e. parameters integrated out). This you can't compute if you have an improper prior. Thus, if it is a problem or not depends on your objective. There is a related question over at Cross-Validated, which I would recommend: https://stats.stackexchange.com/questions/35789/bayes-factors-with-improper-priors