Sub Gaussian concentration of correlated random variables

563 Views Asked by At

Let $X_1, X_2, \dots, X_n$ be $n$ zero mean random variables. It is given that $X_i$ is sub-Gaussian with parameter $\sigma_i^2$ (assumed known) for $i = 1,2, \dots, n$. Consider the random variable $Y = \sum_{i = 1}^n X_i$. It is known that $Y$ is a sub-Gaussian random variable. We denote the corresponding sub-Gaussian parameter as $\sigma_Y^2$.

If $X_i$'s are given to be independent, then it is well-known that $\sigma_Y^2 = \sum_{i = 1}^n \sigma_i^2$. Similarly, a well known upper bound for the general case is $\sigma_Y^2 = \left( \sum_{i = 1}^n \sigma_{i}\right)^2$. However, when the random variables $X_i$'s are correlated such that the covariance matrix is known, can this bound on $\sigma_Y$ be improved using this information? In particular, if we know that the random variables are negatively correlated for all $i$, I would expect the bound on $\sigma_Y$ to be closer to that obtained in the independent case than the one obtained in the general case.

I was wondering if there was any way to prove the same or any standard results that account for correlation between random variables. I was also interested in the case where the random variables are sub-Exponential. Any leads or references will be appreciated. Thanks!

1

There are 1 best solutions below

3
On

Here is a special case for Gaussians. I'm sceptical whether such a nice result can hold very generally.

If $X\sim \mathcal{N}(0,\sigma^2)$, its sub-gaussian norm is $||X||_{\psi_2}=\sigma$. If $(X_1,\dots,X_n)$ is jointly Gaussian-distributed, then $\sum_{i=1}^n X_i$ is also Gaussian. So, you can determine its sub-gaussian norm by analyzing the variance of $\sum_{i=1}^n X_i$. Here, the covariance matrix will indeed show up.

For example, if $(X,Y)$ are jointly Gaussian distributed with mean zero, then: $$ ||X+Y||_{\psi_2}=\sqrt{Var[X+Y]}=\sqrt{E[X^2]+E[Y^2]+2E[XY]}=\sqrt{\sigma^2_X+\sigma^2_Y+\rho_{XY}\sigma_X\sigma_Y} $$ Note that for $\rho_{XY}=0$, we have $||X+Y||_{\psi_2}^2=||X||_{\psi_2}^2+||Y||_{\psi_2}^2$ and for $\rho_{XY}=1$, $||X+Y||_{\psi_2}^2=(||X||_{\psi_2}+||Y||_{\psi_2})^2$. So, for the Gaussian case, we can indeed nicely interpolate between the two extreme cases you describe. As stated in the beginning, I doubt that the general case can be this terrific.