What does rotational invariance mean in statistics?

Question

What does rotational invariance mean in statistics? The property that the normal distribution satisfies for independent normal distributed $X_i$, $\Sigma_i X_i$ is also normal with variance $\Sigma_i Var(X_i)$ is referred to as rotational invariance and I want to know why.

Answer 1

Let $X_i$ have mean $\mu_i$ and standard deviation $\sigma_i$, and write $X_i = \mu_i + \sigma_i Z_i$ where $Z_i$ are independent standard normal random variables. The joint density of $Z_1, \ldots, Z_n$ is $$f(z_1, \ldots, z_n) = (2 \pi)^{-n/2} e^{-(z_1^2 + \ldots + z_n^2)/2} = (2 \pi)^{-n/2} e^{-\|{\bf z}\|^2/2}$$ which is rotationally invariant, i.e. invariant under rotations of $n$-dimensional space, because it only depends on the length of the vector ${\bf z} = (z_1, \ldots, z_n)$. Now $Z_1 = (1,0,\ldots, 0) \cdot \mathbb (Z_1, \ldots, Z_n)$ has a standard normal distribution. But because of the rotational invariance, so does ${\bf u} \cdot (Z_1, \ldots, Z_n)$ for any unit vector $\bf u$.
In particular, take ${\bf u} = (\sigma_1, \ldots, \sigma_n)/\sigma$ where $\sigma = \sqrt{\sigma_1^2 + \ldots + \sigma_n^2}$, and we get that $(\sigma_1 Z_1 + \ldots + \sigma_n Z_n)/\sigma$ has a standard normal distribution, which means $$X_1 + \ldots + X_n = (\mu_1 + \ldots + \mu_n) + \sigma \dfrac{\sigma_1 Z_1 + \ldots + \sigma_n Z_n}{\sigma}$$ has a normal distribution with mean $\mu_1 + \ldots + \mu_n$ and standard deviation $\sigma$.