$2D$ Gaussian Density Reducing to $1D$ Case as Correlation Approaches $1$

Question

I was asked the following question and thought it was interesting.

Is there an obvious way in which we can show the density of a 2D Gaussian "collapses" to a 1D density as the correlation $\rho \rightarrow 1.$ The intuition is that by having two variables $X$ and $Y$ that are perfectly correlated, we should see a function of just one of the random variables (since two of the same variable is redundant). For simplicity, we can take $\mathbb{E}(X) = \mathbb{E}(Y) = 0.$

More precisely, of course we cannot take $\rho = 1$ directly, since this makes the covariance matrix $\Sigma$ singular. But if we write the correlation as say $\rho = 1 - \epsilon$ and take $\epsilon \rightarrow 0$, can we see such a "collapse" in the density to the case of a single Gaussian random variable? I messed around with this for a while, but I couldn't get it to work.

Answer 1

Note that there is no reason why the covariance matrix cannot be singular. The only difference is that the distribution will not have a density with respect to the 2d Lebesgue measure. However, the general definition of a multivariate Gaussian distribution is a random vector $Z$ such that $\langle Z,v \rangle$ has a (1D) Gaussian distribution, for any vector $v$, which does assume anything about the covariance matrix. In particular, if $\rho=1$, then the $x$ and $y$ coordinates are perfectly linearly related, which implies that the distribution is of the form $Z=(X,aX+b)$ for $X$ a 1D Gaussian and some constants $a$ and $b$. Note that this distribution is concentrated on a line, and is multivariate gaussian because $\langle Z,v \rangle=(v_1+av_2)X+bv_2$ is Gaussian, for any $v\in\mathbb{R}^2$.

Answer 2

While it's true that as $\rho\to1$ the random variables $X$ and $Y$ get closer to each other (in the sense $P(|X-Y|>\epsilon)\to 0$), it is not the case that the 2D density converges to a 1D density, because the 2D density remains a function of two variables $(x,y)$ and is constrained to integrate to $1$.

As $\rho\to1$ the bulk of the 2D density gets narrower, concentrating nearer and nearer to the line $y=x$, but at the same time the 2D density gets higher and higher, since its peak value is $\dfrac1{2\pi\sigma_x\sigma_y\sqrt{1-\rho^2}}$. So as a function of two variables, the shape of the joint density is getting more and more extreme as $\rho\to1$. (But the marginal densities of $X$ and $Y$ remain unchanged.)