Conditional variance of sum of two correlated random variables

Question

Let $\theta\sim\mathcal{N}(\bar{\theta},1/\tau_\theta)$ be a normally distributed random variable, $\varepsilon\sim\mathcal{N}(0,1/\tau_\varepsilon)$ be a normally distributed noise term independent from $\theta$, and let $P$ be an affine function of $\theta$ and $\varepsilon$, i.e., $P(\theta,\varepsilon)=a+b\theta+c\varepsilon$ with $b,c\neq 0$.

I want to determine the conditional variance $Var(\theta+P|P=p)$.

I know how to determine the conditional variance $Var(\theta|P=p)$, which should be $Var(\theta|P)=b^2/\tau_\theta+c^2/\tau_\varepsilon$, since $\theta$ and $\varepsilon$ are uncorrelated. I am not entirely sure about $Var(\theta+P|P=p)$ though, since $P$ and $\theta$ are not uncorrelated.

Here is a guess: \begin{equation} Var(\theta+P|P=p)=Var(\theta|P=p)+Var(P|P=p)+2Cov(\theta,P|P=p)\\ =Var(\theta|P=p)+2Cov(\theta,P|P=p) \end{equation} Now I am not sure; since conditional on $P=p$, $P$ does not vary, the covariance term $$2Cov(\theta,P|P=p)=2E[(\theta-E[\theta|P=p])(P-E[P|P=p])|P=p]$$ should be zero, which implies $Var(\theta+P|P=p)=Var(\theta|P=p)$.

Another approach that seems plausible would be \begin{equation} Var(\theta+P|P=p)=Var(\theta+a+b\theta+c\varepsilon|P=p)\\ =Var(\theta(1+b)+c\varepsilon|P=p)\\ \neq Var(\theta|P=p) \end{equation}

Which is it, or is it something else entirely? If it is the latter, what is $Var(\theta(1+b)+c\varepsilon|P=p)$ explicitly? Thanks!

Answer 1

Your first derivation leading to $Var(\theta+P|P=p)=Var(\theta|P=p)$ is correct. Your second derivation is also correct except for the final statement $$Var(\theta+P|P=p) = Var(\theta(1+b)+c\varepsilon|P=p) \implies Var(\theta+P|P=p) \neq Var(\theta|P=p)$$ They could be equal, although they don't appear to be. In fact they are. Here is how to reason it out. You have a relationship between $p, \varepsilon$ and $\theta$: $$P(\theta,\varepsilon)=a+b\theta+c\varepsilon$$ Since $a$ and $b$ are constants, with knowledge of $p$, you also know what the value of the sum $b\theta+c\varepsilon$ is. Now $$Var(\theta(1+b)+c\varepsilon|P=p) = Var(\theta+(b\theta+c\varepsilon)|b\theta+c\varepsilon) = Var(\theta | b\theta+c\varepsilon) = Var(\theta | p)$$ as desired.

Answer 2

It is not true that $Var(\theta|P)=b^2/\tau_\theta+c^2/\tau_\varepsilon$ in these settings. Take, for example, $\epsilon\sim N(0,1)$, $P\{v=-1\}=P\{v=1\}=1/2$ (independent of $\epsilon$), and $\theta=v\epsilon$. Then

$$P\{\theta\le t\}=\frac{1}{2}P\{\epsilon\le t\}+\frac{1}{2}P\{\epsilon\ge -t\}=\Phi(t)$$

so that $\theta\sim N(0,1)$ and $Cov(\epsilon, \theta)=0$. However, $\theta$ is not independent of $\epsilon$.

Assume that $a=0$, $b=c=1$. Then $Var(\theta|P=p)=0$ for any $p\ne 0$ ($P\ne 0\Rightarrow v=1$ so that $P=2\epsilon$ and $\theta=\epsilon$).

---

You need to specify the joint distribution of $(\theta,\epsilon)$ either explicitly or by assuming that $\theta$ and $\epsilon$ are independent (not just uncorrelated).

---

In any case,

$$Var(\theta+P|P)=\mathbb{E}[(\theta+P)^2|P]-(\mathbb{E}[\theta+P|P])^2$$ $$=\mathbb{E}[\theta^2|P]-(\mathbb{E}[\theta|P])^2=Var(\theta|P)$$