Probability, Joint Distributions, Standard Normal

Question

I'm working through a course in Probability (2nd/3rd year) and would like to clarify some idea on joint distributions.

Suppose for example we have independent random variables $(Z_1,Z_2)$ from a distribution, which we will take to be standard normal, i.e. $N[0,1]$, and then we define variables $X(Z_1, Z_2)$ and $Y(Z_1, Z_2)$, functions of $Z_1,Z_2$, how do we find the marginal distributions of $X$ and $Y$ and their joint distribution?

If we look at a specific case, say for example $X=Z_1+Z_2$ and $Y=2Z_1+Z_2$, how could we find the conditional expectation of $Y$ say if we fix a value for $X$, so say $E[Y|X=\alpha]$ for some $\alpha >0$?

I would be very appreciative of anyone who could help clarify these ideas. Best, MM.

Answer 1

Where you are dealing with sums of independent normally distributed random variables (possibly with different variances) then you can find a discussion in the question and answer of stats.stackexchange.com/questions/9071/

For your specific question, you can use this to see $E[Z_1|Z_1+Z_2=\alpha] = \alpha/2$ so $$E[Y|X=\alpha]= E[2Z_1+Z_2|Z_1+Z_2=\alpha]=3\alpha/2,$$ which is fairly intuitive.

You can also use this to see $E[2Z_1|2Z_1+Z_2=\beta] = 4\beta/5$, so $E[-Z_1|2Z_1+Z_2=\beta] = -2\beta/5,$ so $$E[X|Y=\beta]=E[Z_1+Z_2|2Z_1+Z_2=\beta] = 3\beta/5,$$ which I think is less intuitive.

Answer 2

You have raised multiple questions.

• If $Z_1$ and $Z_2$ are jointly normal random variables (not necessarily independent, and not necessarily standard), then $aZ_1 + bZ_2$ and $cZ_1 + dZ_2$ also are jointly normal random variables. Their means, variances, and covariances are given by the usual formulas: $$\begin{align*} E[aZ_1 + bZ_2] &= aE[Z_1] + b[Z_2]\\ E[cZ_1 + dZ_2] &= cE[Z_1] + d[Z_2]\\ \operatorname{var}(aZ_1 + bZ_2) &= a^2\operatorname{var}(Z_1) + b^2\operatorname{var}(Z_2) + 2ab\operatorname{cov}(Z_1,Z_2)\\ \operatorname{var}(cZ_1 + dZ_2) &= c^2\operatorname{var}(Z_1) + d^2\operatorname{var}(Z_2) + 2cd\operatorname{cov}(Z_1,Z_2)\\ \operatorname{cov}(aZ_1 + bZ_2, cZ_1 + dZ_2) &= ac\operatorname{var}(Z_1) + bd\operatorname{var}(Z_2) + (ad+bc)\operatorname{cov}(Z_1,Z_2) \end{align*}$$

  • Jointly normal random variables are also marginally normal random variables. See here for a simple proof that $aZ_1 + bZ_2$ is a normal random variable when $Z_1$ and $Z_2$ are independent standard normal random variables.

  • If $Z_1$ and $Z_2$ are jointly continuous (which happens only when the magnitude of their correlation coefficient is strictly smaller than $1$, that is, $Z_1$ and $Z_2$ are not perfectly correlated), and if $ad-bc \neq 0$, then $aZ_1 + bZ_2$ and $cZ_1 + dZ_2$ are also jointly continuous and their joint density can be expressed in the usual form of the bivariate joint normal density in which the five quantities shown above occur as parameters.

  • If $Z_1$ and $Z_2$ are perfectly correlated, then so are $aZ_1 + bZ_2$ and $cZ_1 + dZ_2$. Thus, $aZ_1 + bZ_2$ and $cZ_1 + dZ_2$ are not jointly continuous, and their joint density function is degenerate (it is a line density) and cannot be expressed in the usual form of the bivariate joint normal density function.

  • If $ad-bc = 0$, then $aZ_1 + bZ_2$ and $cZ_1 + dZ_2$ are perfectly correlated, are not jointly continuous,and and their joint density function is degenerate (it is a line density) and cannot be expressed in the usual form of the bivariate joint normal density function.

• If $X$ and $Y$ are jointly normal random variables, then the conditional distribution of $Y$ given $X$ is a normal distribution with mean $$E[Y\mid X = \alpha] = E[Y] + \left.\left.\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(X)}\right(\alpha - E[X]\right)$$ and variance $$\operatorname{var}(Y \mid X = \alpha) = \operatorname{var}(Y) - \frac{[\operatorname{cov}(X,Y)]^2}{\operatorname{var}(X)}$$ Note that if $X$ and $Y$ are perfectly correlated, then $\operatorname{var}(Y \mid X = \alpha) =0$.