Identifying a Better Estimator for the Three Day Average

Question

Here is a question I am currently working on. I need some help on part b.

An environmental variable $X$ has value $X_i$ on day $i$. Due to budgetary constraints, $X$ is measured only every third day. If $X$ is measured on day $i$, then the observed value of $X_i$ is used to estimate the three day average $$\bar{X}=\frac{1}{3}(X_{i-1}+X_{i}+X_{i+1})$$ We're given that the variables $X_i$ have a multivariate normal distribution with common mean $m$ and common variance $\sigma^2$ and that the correlation coefficient between $X_{i}$ and $X_{j}$ is $\rho_{ij}=0.9^{|i-j|}$.

Part a) Find the distribution of $\bar{X}|X_i=x$

Part b) Based on the answer to a), suggest a better estimator of $\bar{X}$ from $X_i$

Here is how I approached part a). Notice $$E(\bar{X}|X_i=x)=\frac{2m+x}{3}$$ and $$V(\bar{X}|X_i=x)=\frac{1}{9}V(X_{i-1}+X_{i+1}+x)=\frac{181}{450}\sigma^2$$ Hence $\bar{X}|X_i=x \sim N\big(\frac{2m+x}{3},\frac{181}{450}\sigma^2\big)$. Is this correct? If it is, how I can use this result to find a better estimator for the three day average? If we don't have any other information than the observed value on day $i$, I don't see what else we can use as an estimator for $\bar{X}$.

Thank you!

Answer 1

For simplicity, I will call the r.v. $X_1, X_2, X_3$.

Let $a=0.9$, the correlation matrix is given by $$ \Sigma = \sigma^2\begin{bmatrix} 1& a&a^2 \\ a & 1 & a \\ a^2 & a & 1 \end{bmatrix} $$

and is positive-definite, by the spectral theorem we can write $$ \Sigma = S D^{\frac{1}{2}} (D^{\frac{1}{2}})^T S^T = AA^T$$ where $A = S D^{\frac{1}{2}}$, $S$ is orthogonal and $D$ is diagonal with $D = diag(\lambda_1, \lambda_2, \lambda_3)$ (see the actual values here).

So, $X = AZ+m$, where $Z = (Z_1, Z_2, Z_3)$, with $Z_i \sim N(0,1)$ and all independent.

Notice that if $S$ has rows $s_i$, then: $$ X_i = \langle s_i, D^{\frac{1}{2}}Z \rangle + m $$

This poses a linear relation between $Z_1$, $Z_2$ and $Z_3$. If we force $X_i = x_i$ then we can pick some $Z_i$ and write it as a linear function of the others. Suppose that $i=2$ (that is your case, since you have only the middle value), then $Z_2 = \alpha Z_1 + \beta Z_3 + \gamma$ for some coefficients $\alpha, \beta, \gamma$ that depends only on $a$, $\sigma$ and $x_2$. That gives you the distribution of $\mathbb{E}(X | X_2=x_2)$ since $X = AZ+m$ and $$ \mathbb{E}(Z|X_2=x_2) = \begin{bmatrix} Z_1 \\ \alpha Z_1 + \beta Z_3 + \gamma \\ Z_3 \end{bmatrix} $$ with $Z_1, Z_3 \in N(0,1)$ independent.

We want to compute $\bar{X}| X_2=x_2$, this is just: $$ \mathbb{E}(\bar{X}|X_2=x_2) = 3m+\sum_{i=1}^3 \langle s_i, D^{\frac{1}{2}}\mathbb{E}(Z|X_i=x_i) \rangle $$

The above relation can be used to construct an estimative, since: $$ \mathbb{E}(\mathbb{E}(\bar{X}|X_i=x_i)) = 3m+\sum_{i=1}^3 \langle s_i, D^{\frac{1}{2}} \begin{bmatrix} 0 \\ \gamma \\ 0 \end{bmatrix}\rangle = 3m + \sqrt{\lambda_2}\langle s_2, \begin{bmatrix} 0 \\ \gamma \\ 0 \end{bmatrix}\rangle $$

Now you can compute $\lambda_2$, $s_2$ and $\gamma$ (notice that $\gamma$ is a function of $x_2$) numerically or even get the closed expressions with a good CAS. And so you have a better estimative then before.

The good thing about this resolution is that if you want to consider also other measures you can adapt it to improve your estimative.

Answer 2

Is there a flaw in my original argument? Here's another example I saw on another Stack Exchange post to demonstrate the technique I presented. Suppose $X,Y,Z \sim N(0,1)$ are i.i.d and set $W:=\frac{X+ZY}{\sqrt{1+Z^2}}$. Suppose we want to find the distribution of $W$. If we fix an outcome $\{Z=z\}$ we immediately realize that $W$ is a sum of two scaled normal random variables $$W=\Bigg(\frac{1}{\sqrt{1+z^2}}\Bigg)X+\Bigg(\frac{z}{\sqrt{1+z^2}}\Bigg)Y$$ This shift in perspective allows us to recognize that $W|Z=z \sim N(0,1)$ because $$E(W|Z=z)=\Bigg(\frac{1}{\sqrt{1+z^2}}\Bigg)E(X)+\Bigg(\frac{z}{\sqrt{1+z^2}}\Bigg)E(Y)=0$$ and $$V(W|Z=z)=\Bigg(\frac{1}{\sqrt{1+z^2}}\Bigg)^2V(X)+\Bigg(\frac{z}{\sqrt{1+z^2}}\Bigg)^2V(Y)=1$$ We can use the total law of probability to find show that $W \sim N(0,1).$ Indeed, $$f_W(w)=\int_{-\infty}^{\infty}f_{W|Z=z}(w|z)f_{Z}(z)dz=\frac{1}{\sqrt{2\pi}}e^{-w^2/2}$$ So I guess I'm wondering why mentally "freezing" an outcome of $X_i$ and recognizing that $$\bar{X}|X_i=x \sim N\Bigg(\frac{2m+x}{3},\frac{181}{450}\sigma^2\Bigg)$$ isn't valid? Is it because $X_i$ is not independent of $X_{i+1}$ and $X_{i-1}$?

Answer 3

The conditional variance or expectation are not the usual variance or expectation. We need to be careful with independence.

Another (an easy one) solution:

Let $p=0.9$.

We have: $$ E(X_{i-1}+X_i+X_{i+1}|X_i) = E(X_{i_1}| X_i) + E(X_i | X_i) + E(X_{i+1}| X_i)$$

Since $X_i$ is $\sigma(X_i)$-measurable we have $E(X_i | X_i) = X_i$. Now we recall that if $Y_1,Y_2 \sim N(0,1)$ are two gaussian r.v. with correlation $p$, then exists $U, U_1, U_2 \sim N(0,1)$ i.i.d. such that $$ Y_1 = \sqrt{p}U + \sqrt{1-p}U_1\text{ and }Y_2 = \sqrt{p}U + \sqrt{1-p}U_2 $$

Therefore, we can find $U, U_{i}, U_{i+1}$ such that: $$ \frac{X_{i}-m}{\sigma} = \sqrt{p}U + \sqrt{1-p}U_{i}\text{ and }\frac{X_{i+1}-m}{\sigma} = \sqrt{p}U + \sqrt{1-p}U_{i+1}$$

So, $$ E(X_{i+1}| X_i) = \sigma E\left( \sqrt{p}U + \sqrt{1-p}U_{i+1} | X_i \right) + m = \sigma E\left( \sqrt{p}U | X_i \right) + m = \sigma \frac{\sqrt{p}}{\sqrt{1-p}+\sqrt{p}}\frac{X_i-m}{\sigma} + m, $$ here I used the symmetry of $X+Y = E(X+Y|X+Y) = E(X|X+Y) + E(Y|X+Y)$, but you can compute this using the fact that $U, U_{i+1}$ are independent.

The case $X_{i-1}$ is analogous. Then: $$ E(X_{i-1}+X_i+X_{i+1}|X_i) = 2m + 2\frac{\sqrt{p}}{\sqrt{1-p}+\sqrt{p}}(X_i-m) + X_i $$

And so a good estimative for $\bar{X}$ is $$ \bar{X} = \frac{ 2m+2\lambda(x-m) + x }{3} ,$$ where $\lambda = \frac{\sqrt{p}}{\sqrt{1-p}+\sqrt{p}} = \frac{\sqrt{p}}{\sqrt{1-p}+\sqrt{p}}$.

Notice that, for example, if $p$ is high, then $\lambda$ is close to one and $\bar{X}$ is close to $x$. If $p$ is low, then $\lambda$ is close to zero and the estimative becomes the original one.