upper bound of 2-norm of a p dimensional Gaussian random variable?

Question

For $n$ iid observations $\{X_i\}_{i=1}^n$ sampling from $p$ dimensional Gaussian distribution $N(\mu,\Sigma)$, where $\Sigma>0$ and its eigenvalues are bounded away from zero and infinity, the average is $\hat{\mu}=\frac{1}{n}\sum_{i=1}^n X_i$. Then as $n \to \infty$, what's the upper bound of $\|\hat{\mu}-\mu\|_2$ and $\|\hat{\mu}-\mu\|_\infty$? Does the results need other restrictions on $\Sigma$?

Answer 1

For the $L^2$ norm, note that $\mathbb{E}[\hat{\mu}] = \mu,$ so $||\hat{\mu}-\mu||_2^2$ is simply the trace of the variance: $$tr\left[Var(\hat{\mu})\right] = tr(\Sigma/n) = tr(\Sigma)/n.$$ Hence, $||\hat{\mu}-\mu||_2 = \sqrt{tr(\Sigma)}/\sqrt{n}.$ This is an exact result for any $n.$

For the $L^\infty$ norm, we always get $||\hat{\mu}-\mu||_\infty=\infty,$ unless $\Sigma=0.$ Since if $\Sigma\neq 0$, then for any $n$ and any $K>0,$ $$\mathbb{P}(||\hat{\mu}-\mu||>K) > 0,$$ since $\hat{\mu}\sim N(\mu, \Sigma/n).$

If $\Sigma = 0,$ we have $\hat{\mu}=\mu$ a.s., so $||\hat{\mu}-\mu||_\infty=0.$