Given a $k$-variate normal distribution with mean $\mathbf \mu$ and covariance matrix $\mathbf \Sigma$, what is the variance of a 1-draw sample (which will be of size $k$) from this distribution? (I am not even sure how to begin calculating this.)
Add 1
So, given one such draw $\mathbf X = (X_1,X_2,...,X_k)^T$, the sample variance (or variance of the sample?) is given by
$$\frac{1}{k}\sum_{i=1}^{k}\left({X_i}-\frac{1}{k}\sum_{i=1}^{k}{X_i} \right)^2$$
but what about "population"? Do we "just" take the expectation of the above?
Add 2
If we expand the above to express it as
$$\frac{1}{k}\sum_{i=1}^{k}\left({X_i}\right)^2 - \left(\frac{1}{k}\sum_{i=1}^{k}{X_i} \right)^2$$
then the second term is a square of a normal RV with mean $m = \frac{1}{k}\sum_{i=1}^{k}{\mu_i}$ and variance (if I am not mistaken) $\sigma^2=\frac{1}{k^2} \mathbf 1^T \mathbf \Sigma \mathbf 1$.
Clearly,
$$\left( X_i - \frac{1}{k} \sum_{j=1}^k X_j \right)^2 = X_i^2 - \frac{2}{k} \sum_{j=1}^k X_i X_j + \frac{1}{k^2} \sum_{j=1}^k \sum_{\ell=1}^k X_j X_{\ell}.$$
Summing over $i=1,\ldots,k$ and taking the expectation we find
$$\begin{align*} \mathbb{E} \left( \frac{1}{k} \sum_{i=1}^k \left( X_i - \frac{1}{k} \sum_{j=1}^k X_j \right)^2 \right) &= \frac{1}{k} \sum_{i=1}^k \mathbb{E}(X_i^2) - \frac{2}{k^2} \sum_{i=1}^k \sum_{j=1}^k \mathbb{E}(X_i X_j) + \frac{1}{k^2} \sum_{j=1}^k \sum_{\ell=1}^k\mathbb{E}(X_j X_{\ell}). \\ &= \frac{1}{k} \sum_{i=1}^k \mathbb{E}(X_i^2) - \frac{1}{k^2} \sum_{i=1}^k \sum_{j=1}^{k} \mathbb{E}(X_i X_j). \end{align*}$$
The right-hand side can be expressed in terms of the covariance matrix $\Sigma$ and the mean $\mu$.