Confidence interval for mean based on a single trajectory of a first-order autoregressive process

Question

I am currently studying Statistics for Spatial Data, revised edition, by Cressie. Chapter 1.3 STATISTICS FOR SPATIAL DATA: WHY? says the following:

1.3 STATISTICS FOR SPATIAL DATA: WHY?

Some simple spatial models will be given to show the effect of correlation on estimation, prediction, and design. The models allow closed-form expressions to be calculated, from which discussion of more general issues can be initiated.

Estimation
Consider the following simple statistical model, taught in all beginning Statistics service courses. Suppose $Z(1), \dots, Z(n)$ are independent and identically distributed (i.i.d.) from a Gaussian (i.e., normal) distribution with unknown mean $\mu$ and known variance $\sigma_0^2$. The minimum-variance unbiased estimator of $\mu$ is

The estimator of $\mu$ is $$\bar{Z} \equiv \sum_{i=1}^{n} \frac{Z(i)}{n} \tag{1.3.1}$$

and inference on $\mu$ is straightforward: The estimator $\bar{Z}$ is Gaussian with mean $\mu$ and variance $\dfrac{\sigma_0^2}{n}$. Thus, a two-sided 95% confidence interval for $\mu$ is

$$\left(\bar{Z} - (1.96)\frac{\sigma_0}{\sqrt{n}}, \bar{Z} + (1.96)\frac{\sigma_0}{\sqrt{n}}\right). \tag{1.3.2}$$

Instead of independent data, now suppose the data are positively correlated with a correlation that decreases as the separation between data increases:

$$\text{cov}(Z(i), Z(j)) = \sigma_0^2 \cdot \rho^{\lvert i - j \rvert}, \quad i, j = 1, \dots, n, \quad 0 < \rho < 1. \tag{1.3.3}$$

It is well known that such a correlation function results from a first-order autoregressive process $Z(i) = \rho Z(i - 1) + \varepsilon(i)$, $i \geq 1$, where $\varepsilon(i)$ is part of an i.i.d. sequence of Gaussian random variables with zero mean and variance $\sigma_0^2 (1 - \rho^2)$ and is independent of $Z(i - 1)$; see, e.g., Fuller (1976), Section 2.3. However, this is not the generating mechanism I have in mind; here, $\mathbf{Z} \equiv (Z(1), \dots, Z(n))^\prime$ are spatial data in $\mathbb{R}^1$ from which prediction of $Z(0)$ or $Z(3/2)$ is just as appropriate as that of $Z(n + 1)$.

Now, $$\begin{align} \text{var}(\bar{Z}) &= n^{-2} \left[\sum_{i=1}^{n} \sum_{j=1}^{n} \text{cov}(Z(i), Z(j))\right] \\ &= \left( \frac{\sigma_0^2}{n}\right) \left\{ 1 + 2 \left[ \left( \frac{\rho}{1 - \rho} \right) \left(1 - \frac{1}{n}\right) \right] - 2\left[ \left( \frac{\rho}{1 - \rho} \right)^2 \left( \frac{1 - \rho^{n-1}}{n} \right) \right] \right\}. \tag{1.3.4} \end{align}$$

For $n = 10$ and $\rho = 0.26$, $\text{var}(\bar{Z}) = \frac{\sigma_0^2}{10}[1.608]$, and a two-sided 95% confidence interval for $\mu$ is

$$\left(\bar{Z} - (2.485)\frac{\sigma_0}{\sqrt{10}}, \bar{Z} + (2.485)\frac{\sigma_0}{\sqrt{10}}\right).$$

Thus, failure to realize the presence of positive correlation in the data leads to a confidence interval (1.3.2) that is too narrow; for $n = 10$ and $\rho = 0.26$, the actual coverage probability of (1.3.2) is 87.8%, not 95%. [If $-1 < \rho <0$, (1.3.2) would be too wide, but because positive spatial dependence is seen more often, these introductory remarks concentrate on situations where $\rho$ is positive.]

There are two parts of this that I don't understand:

1. The derivation of (1.3.4).
2. Where the 2.485 came from in the confidence interval.

I would like help with both of these.

---

My work for 1.

How do we deal with $\sum_{i=1}^{n} \sum_{j=1}^{n} \text{cov}(Z(i), Z(j))$ in $\text{var}(\bar{Z}) = n^{-2} \left[\sum_{i=1}^{n} \sum_{j=1}^{n} \text{cov}(Z(i), Z(j))\right]$?

For the $i = j$ terms, we have $n \sigma_0^2$.

For the $i \not= j$ terms, we use the formula for finite geometric series.

Begin by noticing the following:

• For a distance of 1, there are $n - 1$ pairs like $(Z(1), Z(2)), (Z(2), Z(3)), \dots, (Z(n-1), Z(n))$, and also $n - 1$ pairs in the opposite direction, like $(Z(2), Z(1)), (Z(3), Z(2)), \dots, (Z(n), Z(n-1))$, resulting in $2(n - 1)$ pairs in total.
• For a distance of 2, there are $n - 2$ pairs one way and $n - 2$ the other, so $2(n - 2)$ pairs in total.
• This pattern continues until the maximum distance of $n - 1$, where there is only 1 pair in each direction, so 2 pairs in total.

So for the $i \not= j$ case of the summation, we have $2 \sum_{k = 1}^{n - 1} (n - k) \rho^k \sigma_0^2$. So, overall, we have $\text{var}(\bar{Z}) = n^{-2} \left[ n \sigma_0^2 + 2 \sum_{k = 1}^{n - 1} (n - k) \rho^k \sigma_0^2 \right]$.

Now let us focus on $$\sum_{k = 1}^{n - 1} (n - k) \rho^k \sigma_0^2 = \sigma_0^2 \sum_{k = 1}^{n - 1} (n - k) \rho^k = \sigma_0^2 \left( n \sum_{k = 1}^{n - 1} \rho^k - \sum_{k = 1}^{n - 1} k \rho^k \right)$$

The finite geometric series formula is $S = a \left( \dfrac{1 - r^n}{1 - r} \right)$. Applying this to $n \sum_{k = 1}^{n - 1} \rho^k$: $$n \left[ \rho \left( \dfrac{1 - \rho^{n - 1}}{1 - \rho} \right) \right] = n \left( \dfrac{\rho - \rho^n}{1 - \rho} \right)$$ Applying it to $\sum_{k = 1}^{n - 1} k \rho^k$: $$\rho \dfrac{d}{d \rho} \left( \dfrac{1 - \rho^{n - 1}}{1 - \rho} \right) = \rho \left[ \dfrac{-(n - 1)\rho^{n - 2}(1 - \rho) - (1 - \rho^{n - 1})(-1)}{(1 - \rho)^2} \right] = \dfrac{-(n - 1) \rho^{n - 1}(1 - \rho) + (\rho - \rho^n)}{(1 - \rho)^2}$$

So we should have that $\sum_{k = 1}^{n - 1} (n - k) \rho^k \sigma_0^2 = \sigma_0^2 \sum_{k = 1}^{n - 1} (n - k) \rho^k = \sigma \left( n \sum_{k = 1}^{n - 1} \rho^k - \sum_{k = 1}^{n - 1} k \rho^k \right) = \sigma_0^2 \left\{ n \left( \dfrac{\rho - \rho^n}{1 - \rho} \right) + \dfrac{(n - 1) \rho^{n - 1}(1 - \rho) - (\rho - \rho^n)}{(1 - \rho)^2} \right\}$

So, overall, we have that $$\text{var}(\bar{Z}) = n^{-2} \left\{ n \sigma_0^2 + 2 \left\{ \sigma_0^2 \left[ n \left( \dfrac{\rho - \rho^n}{1 - \rho} \right) + \dfrac{(n - 1) \rho^{n - 1}(1 - \rho) - (\rho - \rho^n)}{(1 - \rho)^2} \right] \right\} \right\}$$

However, so far, I have not been able to get from this to the author's result (1.3.4).

Answer 1

First note that the formula given in (1.3.4) can be simplified as

$$ \fbox{$\text{var}(\bar{Z})=\frac{\sigma_0^2}{n} \left\{ 1 + \frac{2\rho}{1 - \rho} \left(1 - \frac{1-\rho^{n}}{n(1-\rho) } \right) \right\}.$}$$

Hence, the standard deviation of $\bar{Z}$ for correlated data becomes:

$$\text{SD}(\bar{Z})=\frac{\sigma_0}{\sqrt{n}} \sqrt{1 + \frac{2\rho}{1 - \rho} \left(1 - \frac{1-\rho^{n-1}}{n(1-\rho) } \right) } .$$

This is used instead of $\frac{\sigma_0}{\sqrt{n}}$ in (1.3.2).

For $n = 10$ and $\rho = 0.26$,

$$\sqrt{1 + \frac{2\rho}{1 - \rho} \left(1 - \frac{1-\rho^{n-1}}{n(1-\rho) } \right) }=\sqrt{1.607743012}=1.2679680643$$

You can see that

$$ 1.96 \times 1.2679680643 \frac{\sigma_0}{\sqrt{n}}=2.485217406 \frac{\sigma_0}{\sqrt{n}}$$

where $1.96$ is the $z_{0.025}$ already given in (1.3.2) (your second question).

Now I show how we can derive the formula in the box:

$$\begin{align} \frac{n^{2}}{\sigma_0^2} \text{var}(\bar{Z}) &= \sum_{i=1}^{n} \sum_{j=1}^{n} \text{cov}(Z(i), Z(j))\\ &= \sum_{i=1}^{n} \text{cov}(Z(i), Z(i))+2 \sum_{i=1}^{n} \sum_{j=i+1}^{n} \text{cov}(Z(i), Z(j)) \\ &=\sum_{i=1}^{n} \rho^{|i-i|}+2 \sum_{i=1}^{n} \sum_{j=i+1}^{n} \rho^{j-i} \\ &= n+2 \sum_{i=1}^{n} \sum_{k=1}^{n-i} \rho^{k}\\ &=n+2 \sum_{i=1}^{n} \rho \frac{1-\rho^{n-i}}{1-\rho}\\ &=n+2 \frac{\rho}{1-\rho}\sum_{i=1}^{n} (1-\rho^{n-i})\\ &=n+2 \frac{\rho}{1-\rho} (n-\sum_{i=1}^{n}\rho^{n-i})\\ &= n+2 \frac{\rho}{1-\rho} (n-\sum_{l=0}^{n-1}\rho^{l})\\ &=n+2 \frac{\rho}{1-\rho} \left (n-\frac{1-\rho^{n}}{1-\rho} \right ). \end{align}$$

This proves the formula (your first question).