question about correlation of variables

Question

Here is an interview question I had and cannot figure out how to solve it. Any hint?

Let $X$, $Y$, $Z$ be 3 random variables such that $\mathsf{Corr}(X, Y)=0.9$ and $\mathsf{Corr}(Y, Z)=0.8$. What is the minimum correlation between $X$ and $Z$?

Thanks for your help!

Answer 1

This Answer deals with the special case of normal distributions. Without loss of generality in this case, we use distributions with zero means and unit standard deviations. Our purpose is to construct three such normal random variables $X_1, X_2, X_3$ with $Cor(X_1, X_2) = \rho_{12}$ and $Cor(X_2, X_3) = \rho_{23}$. Then we show that $Cor(X_1, X_3) = \rho_{13} = \rho_{12}\rho_{23}$.

We use $X_1 \sim Norm(0,1)$ along with auxilliarly $S_1 \sim Norm(0,1)$ to make $X_2 \sim Norm(0,1)$ such that $Cor(X_1, X_2) = \rho_{12}$, for a given $\rho_{12}.$ Let $$X_2 = \rho_{12}X_1 + \delta_{12}S_1,$$ where $\delta_{12} = \sqrt{1 - \rho_{12}^2}.$ It is elementary to show that $V(X_2) = 1$ and, by the independence of $X_1$ and $S_1$, that $Cov(X_1,X_2) = Cor(X_1,X_2) = \rho_{12}.$

We use $X_2 \sim Norm(0,1)$ along with auxillarly $S_2 \sim Norm(0,1)$ to make $X_3 \sim Norm(0,1)$ such that $Cor(X_2, X_3) = \rho_{23}$, for a given $\rho_{23}.$ Let $$X_3 = \rho_{23}X_2 + \delta_{23}S_2,$$ where $\delta_{23} = \sqrt{1 - \rho_{23}^2}.$ It is also elementary to show that $V(X_3) = 1$ and (as claimed) that $Cor(X_2,X_3) = \rho_{12}\rho_{23}.$

The following simulation in R illustrates these relationships for vectors of length one million for $\rho_{12} = 0.8$ and $\rho_{23} = 0.9.$

 m = 10^6
 rho12 = .8;  dlt12 = sqrt(1 - rho12^2)
 rho23 = .9;  dlt23 = sqrt(1 - rho23^2)

 x1 = rnorm(m);  s1 = rnorm(m)
 x2 = rho12*x1 + dlt12*s1
 mean(x1); sd(x1); mean(x2); sd(x2)
 ## 0.0001394826  
 ## 0.9995465
 ## -0.0008933496
 ## 1.000777
 cor(x1, x2)
 ## 0.8001522

 s2 = rnorm(m)
 x3 = rho23*x2 + dlt23*s2
 mean(x3); sd(x3)
 ## -0.001117523
 ## 1.000685
 cor(x2, x3)
 ## 0.9001839

 ## cor(x1, x3)
 ## 0.7200575
 MAT = cbind(x1,x2,x3)
 pairs(MAT[1:30000,], pch=".")

 cor(MAT)
 ##           x1        x2        x3
 ## x1 1.0000000 0.8001522 0.7200575
 ## x2 0.8001522 1.0000000 0.9001839
 ## x3 0.7200575 0.9001839 1.0000000

Note: For multivariate normal distributions and the given correlations assumed above, I do not claim that $\rho_{13} = 0.72$ is the only possible answer. (There are many alternative ways to simulatie normals with specified correlations. The method I used above is discussed here).

The requirement in the normal case is for the variance-covariance matrix $\mathbf{V}$ to be positive definite. The variance-covariance matrix obtained for this particular method of simulation is indeed positive definite; its eigenvalues are all positive:

 V = matrix(c(1.0,  0.8, 0.72,
              0.8,  1.0, 0.9,
              0.72, 0.9, 1.0), byrow=T, nrow=3)
 eigen(V)$values
 ## 2.6155944 0.2960829 0.0883227

Baaed on a grid search, it seems values of $\rho_{13}$ in $(0.459, 0.981)$ lead to positive-definite $\mathbf{V}$. (There is a function in R to generate multivariate normal distributions with specified parameters.)

Here is another example using the simulation method above. With the single change rho12 = .5, the program above gives output ending as follows:

 .....
 cor(x1, x3)
 ## 0.4500347
 MAT = cbind(x1,x2,x3)
 pairs(MAT[1:30000,], pch=".")

 cor(MAT)
 ##           x1        x2        x3
 ## x1 1.0000000 0.4990733 0.4500347
 ## x2 0.4990733 1.0000000 0.9002335
 ## x3 0.4500347 0.9002335 1.0000000

Answer 2

Comment. Playing around with small, non-normal datasets, I found the following example, for whatever it may be worth. It has $Cor(X, Y) \approx .7,\; Cor(Y, Z) \approx .7$ and $Cor(X, Z) = 0.$

 x = -5:5;  z = x^2;  y = (3*x + z)/2
 DTA = cbind(x,y,z)
 t(DTA)
 ##   [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
 ## x   -5   -4   -3   -2   -1    0    1    2    3     4     5
 ## y    5    2    0   -1   -1    0    2    5    9    14    20
 ## z   25   16    9    4    1    0    1    4    9    16    25
 cor(DTA)
 ##          x         y         z
 ## x 1.000000 0.7319251 0.0000000
 ## y 0.731925 1.0000000 0.6813851
 ## z 0.000000 0.6813851 1.0000000
 pairs(DTA, pch=19)