Degrees of Freedom in a Wishart distribution

Question

What does the degrees of freedom parameter $n$ mean intuitively in a Wishart distribution $\mathcal{W}_p(\mathbf{V},n)$? Does it have any relation to the covariance of different dimensions of the resulting covariance matrix? Why is $n==p$ called a non-informative prior?

Answer 1

If we consider the expectations of a covariance matrix $\Sigma^{-1}$ under out prior assumptions that is follows an inverse-Wishart distribution, we see $E(\Sigma^{-1})=nV$ for inverse covariance matrix $V$. Essentially the degrees of freedom parameter arises from statisticians assuming it is a (positive) integer $\Bbb Z^+$, which means we have a multivariate generalisation of the $\chi_\nu^2$ distribution. Note they do not have to be integers.

You will find that as the Chi-squared distribution describes the sums of squares of $n$ draws from a univariate normal distribution, the Wishart distribution represents the sums of squares (and cross-products) of $n$ draws from a multivariate normal distribution.

Hence intuitively for some dimension $p$, if we want to assume:

(1) We are confident about the true covariance matrix being near some covariance matrix $\Sigma_0$, then we can choose a large value for $n$ say $p+10$ and set $V=[n-p-1]\Sigma_0$, so the distribution of our prior $\Sigma^{-1}$ is concentrated around $\Sigma_0$.

(2) We are not too sure about the true covariance matrix being near some covariance matrix $\Sigma_0$, then we can choose a small value for $n$ say $p+2$ and set $V=\Sigma_0$, so the distribution of our prior $\Sigma^{-1}$ is only loosely centered around $\Sigma_0$.

As $n$ decreases to $p$, the distribution becomes increasingly loosely centered around $\Sigma_0$. Hence it will be non-informative when the degrees of freedom is equal to the dimension/number of parameters. This is also clear since the first moment of a Wishart distribution does not exist unless $n>p+1$, with an improper distribution at $n=p$.

$n$ simply then deals with how much we assume about the distribution of the covaraince. But it also happens that if $\Sigma^{-1}$ follows a Wishart distribution, then so does any sub-matrix. In fact I think it can be shown that the sub-matrix degrees of freedom is independent of $p$, the dimension. So technically the degrees of freedom for the sub-matrix parallels that of the original matrix.