Conditional expectations and weighted sums

Question

Let $X$ and $Z$ denote random variables; and define $Y$ as their weighted average, i.e. = $Y = \lambda X + (1 - \lambda)Z$ for some $\lambda \in [0, 1]$. Assume that $X$ and $Z$ are independently drawn from the same smooth distribution, with mean $\mathbb{E}[X] = \mathbb{E}[Z] = 0$. We are interested in the conditional expectation $\mathbb{E}[X|Y = y]$.

In the extreme case that $\lambda = 0$, $Y$ and $X$ are unrelated so $\mathbb{E}[X|Y = y] = \mathbb{E}[X] = 0$.

In the other extreme case that $\lambda = 1$, we have $Y = X$ so $\mathbb{E}[X|Y = y] = y$.

Given this, it is natural to conjecture that $|\mathbb{E}[X|Y = y]|$ is strictly increasing in $\lambda$. (Indeed, one may think that $\mathbb{E}[X|Y = y] = \lambda y$.) Is this true in general? If not, what are minimal assumptions on the distribution required to make this true?

Answer 1

The conjecture $$ \mathbb E[X|Y=y]=\lambda y $$ is not true in general.

Example.

When $X,Z$ are i.i.d and $N(0,1)$ then $Y$ is $N(0,\sigma^2)$ with $$\tag{1}\sigma^2=\lambda^2+(1-\lambda)^2\,.$$ Since $X$ and $Y$ have correlation $$\rho=\frac{\lambda}{\sigma}$$ their joint density is $$\tag{2} f(x,y)=\frac{1}{2\pi\sigma\sqrt{1-\rho^2}}\exp\Big(-\frac{\sigma^2x^2-2\sigma\rho xy+y^2}{2\sigma^2(1-\rho^2)}\Big)\,. $$ According to this answer $$\tag{3}\boxed{\quad \mathbb E[X|Y=y]=\rho\frac{y}{\sigma}=\frac{\lambda y}{\sigma^2}=\frac{\lambda y}{\lambda^2+(1-\lambda)^2}\,.\quad} $$ The function $\lambda\mapsto\frac{\lambda}{\lambda^2+(1-\lambda)^2}$ is not monotone. It has a maximum of approx. $1.21$ around $\lambda=0.70\,:$

Remark.

When $X,Y$ are i.i.d. and $N(0,1)$ and we change $Y$ slightly to $$\tag{4} Y'=\lambda X+\sqrt{1-\lambda^2}Z $$ then it is easy to see that $Y'$ is $N(0,1)$ and has correlation $\lambda$ with $X\,.$ Like before it follows now that the conjectured relationship $$ \mathbb E[X|Y'=y']=\lambda y' $$ holds because $\sigma^2$ is now one.

Answer 2

More an idea than an answer, but too long for a comment:

Assume that the random $X,Z$ are i.i.d. drawn from a distribution that has probability density $q(x)$.

Let $p(x,y)$ be the joint density of $X$ and $Y$. We have $$p(x|y) = \frac{p(x,y)} {p_y(y)} = \frac{p(y|x) q(x)} {p_y(y)}$$

Given $X=x$, we have that $Y = \lambda x + (1-\lambda) Z$, thus conditional on $X = x$, we have formally $$ \mathbb{P}[Y = y| X = x] = \mathbb{P} [ Z = \frac{y-\lambda x}{1-\lambda}],$$ using independence.

Thus, we get for the conditional density

$$p(y|x) = \frac{1-\lambda}{\lambda}q\left( \frac{y -\lambda x}{1-\lambda}\right).$$

Analagously, we have for the conditional density on $Z$ $$p(y |z) = \frac{\lambda}{1-\lambda}q\left(\frac{y -(1-\lambda)z}{\lambda}\right).$$

Then we get for the conditional expectation

$$\mathbb{E}[X| Y=y] = \int x p(x\vert y) dx = \int x \frac{p(y|x) q(x)} {p_y(y)} dx = \int x \frac{\frac{1-\lambda}{\lambda}q\left( \frac{y -\lambda x}{1-\lambda}\right) q(x)} {p_y(y)} dx = \frac{\int x \frac{1-\lambda}{\lambda}q\left( \frac{y -\lambda x}{1-\lambda}\right) q(x) dx} {\int \frac{1-\lambda}{\lambda}q\left( \frac{y -\lambda x}{1-\lambda} \right) q(x) dx} = \frac{\int x q\left( \frac{y -\lambda x}{1-\lambda}\right) q(x) dx} {\int q\left( \frac{y -\lambda x}{1-\lambda} \right) q(x) dx}. $$

This is now an expression that you could differentiate with respect to $\lambda$, which might yield some insight on the monotonicity (I did not do the calculations, so I don't know if this helps in the end).