Note: I am not asking about the interpretation of covariance as an inner product on the space of random variables.
I have two $n$-dimensional random variables $\vec X, \vec Y\in\mathbb{R}^n$. Each has some covariance matrix $\Sigma_{\vec X}, \Sigma_{\vec Y}$. Each is also "nice", namely
- $\vec X\sim\mathcal{N}(0, \sigma^2I_n)$, and
- $\vec Y\gets \mathsf{Unif}(K)$ for a compact convex set $K$.
I want to compute $\mathsf{Var}(\langle \vec X, \vec Y\rangle)$. Is there a simpler way to do this than via explicitly working with the density of $\langle \vec X, \vec Y\rangle$? For example, is it some function of $\Sigma_{\vec X}, \Sigma_{\vec Y}$?
I assume $X$ and $Y$ are independent (your post doesn't say so, but your title does). Then taking $X\sim N(0,\Sigma_X),Y\sim \text{Unif}(K)$ as column random vectors, we have
$$\begin{align}V(X'Y)&=E[X'YY'X]-(E[X'Y])^2\\ &=E[X'YY'X]-(\underbrace{E[X']}_{=0}E[Y])^2 \quad (\text{by indep})\\ &=\text{tr}E[X'YY'X]\\ &=E[\text{tr}(X'YY'X)]\\ &=E[\text{tr}(XX'YY')]\\ &=\text{tr}E[XX'YY']\\ &=\text{tr}E[XX']E[YY'] \quad (\text{by indep})\\ &=\text{tr}(\Sigma_X(\Sigma_Y+\mu_Y\mu_Y'))\\ \end{align}$$
where $\mu_Y,\Sigma_Y$ are respectively mean and covariance matrix of $Y$.
If $\Sigma_X=\sigma^2I_n,$ then this simplifies to
$$\sigma^2\text{tr}(\Sigma_Y+\mu_Y\mu_Y').$$