Let $Z_i \sim \mathcal{N}(0,1)$ be independent normal distributions. Consider the following correlated variables, defined by $$ X_1 = \frac{Z_1 + Z_2}{\sqrt{2}},\;\;\;X_2= \frac{Z_2 + Z_3}{\sqrt{2}},\;\;\;X_3= \frac{Z_3 + Z_4}{\sqrt{2}},\ldots$$
Thus each $X_i$ by itself is also a standard normal distribution but is correlated to the immediate neighbours $X_{i-1}$ and $X_{i+1}$. Consider the joint distribution of $(X_1,X_2)$ which is a joint normal with mean = $(0,0)$ and covariance matrix $$\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix} $$
Now the thing is according to the rules of conditional probability the conditional variance for $X_1$ is $\left(1-\rho^2\right)\sigma_1^2 = \frac{3}{4}$ in this case. So far so good.
Suppose we then consider the joint normal $\left(X_1,X_2,X_3\right)$, which has the covariance matrix $$\begin{pmatrix} 1 & 1/2 & 0 \\ 1/2 & 1 & 1/2 \\ 0 & 1/2 & 1 \end{pmatrix}$$
In this case, the conditional variance of $X_1$ is given by
$$ 1 - \begin{pmatrix} 1/2 & 0 \end{pmatrix}\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix}^{-1}\begin{pmatrix} 1/2 \\ 0 \end{pmatrix} = 2/3$$
The questions I have now are:
- Since $X_1$ is not dependent on $X_3$ at all, why the conditional variance of $X_1$ drops from $3/4$ to $2/3$ when $X_3$ is taken into account?
- If I further include $X_4,X_5,\ldots$ the conditional variance seems to drop further and reaches a limit of $1/2$ when I include a very large number of $X_i$. Is there any intuitive explanation for this limit?