The dimension of the parameter space of multivariate normal distribution?

Question

In multivariate normal distribution, $X\sim N(\mu,\Sigma)$, where $\mu\in \mathbb{R}^p,~\Sigma \in\mathbb{R}^{p\times p},~ \Sigma \succ0\text{ and }\Sigma$ is symmetric. My professor told me that the dimension of the parameters space is $p+{p+1 \choose 2}$. I know the dimension of $\mu$ is $p$, but I don't know where does the ${p+1 \choose 2}$ come from.

Answer 1

You have $\mu\in\mathbb R^p$ and $\Sigma\in \mathbb R^{p\times p}$ and $\Sigma$ is symmetric.

If there were no other constraints on the parameter space that would give you the number of scalar components of $\mu,$ which is $p,$ plus the number of scalar entries on or above the diagonal of $\Sigma$ (since those below the diagonal are completely determined by those above. The number on the diagonal is $p.$ The number above the diagonal is $\dbinom p 2,$ since you have to choose one of the $p$ rows and one of the $p$ columns, and they can't both have the same index, so you have to choose two of $1,\ldots, p.$ That means you have $$ p + p + \binom p 2. $$ And then $$ p + \binom p 2 = p + \frac{p(p-1)} 2 = \frac{2p}2 + \frac{p^2-p} 2 = \frac{p(p+1)} 2 = \binom {p+1} 2. $$ The subtler part is showing that the constraint that $\Sigma$ must be nonnegative-definite does not further reduce the dimension.

That can be shown by induction on $p$ if you know a certain trick, described in this posting.

PS: I suppose it should also be mentioned that every nonnegative-definite matrix is the variance of some random vector. Thus the constraint that $\Sigma$ must be such a variance does not diminish the parameter space beyond saying $\Sigma$ must be nonnegative-definite. This "every" can be deduced from the finite-dimensional version of the spectral theorem.