Can someone explain Wishart distribution?

Question

I have to use Wishart distribution to model noise in images. Can someone explain or give intuition behind wishart distribution.

Thank you !!!

Answer 1

Suppose $X_1,\ldots,X_n\sim\mathrm{i.i.d.}\ N_p(0,\Sigma)$, i.e. these are independent random variables having a $p$-dimensional normal distribution with expected value $0\in\mathbb R^p$ and variance $\Sigma\in\mathbb R^{p\times p}$. The variance is defined as $$ \Sigma=\operatorname{var}(X) = \operatorname{E}((X-\mu)(X-\mu)^T)\in\mathbb R^{p\times p} $$ where $\mu = \operatorname{E}(X)$. This matrix $\Sigma$ is a non-negative-definite symmetric matrix. Then a Wishart distribution $W_p(n,\Sigma)$ (notations vary) with $n$ degrees of freedom is the probability distribution of the random matrix $$ W=X_1 X_1^T+\cdots+X_n X_n^T. $$ This matrix has expected value $n\Sigma$. If $\Sigma$ has full rank and $n\ge p$ then $W$ has full rank almost surely. If $p=1$ and $\Sigma=1$ then this is a chi-square distribution with $n$ degrees of freedom.

Supppose $\bar X=(X_1+\cdots+X_n)/n$. Then it can be shown that $$ (X_1-\bar X)(X_1-\bar X)^T + \cdots + (X_n-\bar X)(X_n-\bar X)^T \sim W_p(n-1, \Sigma). $$ This sum divided by $n$ is the maximum-likelihood estimate of $\Sigma$, and if divided by $n-1$ it is an unbiased estimate of $\Sigma$.

Suppose $$ \begin{bmatrix} W_{11} & W_{12} \\ W_{21} & W_{22} \end{bmatrix} \sim W_{a+b}\left(n, \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} \right) $$ and $W\in\mathbb R^{a\times a}$, $\Sigma_{11}\in\mathbb R^{a\times a}$, etc. Then the distribution of a Schur complement is given by $$ W_{11} - W_{12} W_{22}^{-1} W_{21} \sim W_a(n,\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}). $$ The distribution of the sum of two independent Wishart matrices with the same $\Sigma$ is another Wishart distribution whose number of degrees of freedom is the sum of the two separate ones.

There is a generalization to non-integer $n$ that also satifies this relation on sums of independent Wishart matrices, but it is defined only for $n\ge p-1$ (if I'm recalling this correctly, but I'm not sure) and for half-integer values of $n$ smaller than that (again if I'm recallying correctly, but again I'm not sure).

That is most of what I recall off the top of my head without looking at old course notes. In other words, there's a lot I don't remember except rather vaguely.

Relevant Wikipedia articles would include not only Wishart distribution but also Estimation of covariance matrices.