How does one represent an econometric model? (an application of the ergodic theorem to stationary time series).

38 Views Asked by At

THE FRAMEWORK:

Let $X_1 , X_2 , \dots, X_n$ be observed random variables. Then one could decide to model them as IID Gaussians with variance one and mean $\mu$.

That is we choose as our model the set of parametric probability density functions given by $$\left \{ f(x| \mu) = \frac{1}{\sqrt{2 \pi }} \exp \left( \frac{-(x-\mu)^2}{2 } \right) \Bigg| \mu \in \mathbb{R} \right \}$$

Then one could find the maximum likelihood estimator for $\mu$ by maximizing the log-likelihood (wrt $\mu$) that is given by

$$\sum_{i=1}^n \log (f(X_i|\mu))$$

We notice that (assuming the true distribution of the IID random variables is $f(x|\mu_0)$, $\mu_0 \in \mathbb{R}$ ) by the law of large numbers

$$\frac{1}{n}\sum_{i=1}^n \log (f(X_i|\mu)) \rightarrow E[\log f(X|\mu) ]$$

where the expectation is with respect to the true density $f(x| \mu_0)$.

This is all well and good but now suppose I choose as my model a stationary AR(1), that is (as Wikipedia puts it)

$$ X_t = \theta X_{t-1} + \epsilon_t $$ where $\epsilon_t \sim N(0,1) \ \forall{t} $ and $|\theta|< 1$.

MY QUESTION:

Being that a statistical model is a set of probability distributions what is the statistical model in this case? Moreover do we obtain an analogous result as the one given by the law of large numbers in the previous case (I suppose through the ergodic theorem applied to stationary processes) ?

MY ATTEMPT:

I would say that the statistical model is defined in terms of the conditional distributions as

$$\left \{ f(x| X_{t-1}, \theta) = \frac{1}{\sqrt{2 \pi }} \exp \left( \frac{-(x-\theta X_{t-1})^2}{2 } \right) \Bigg| \theta \in ]0,1[, X_{t-1} \in \{ X_1, \dots, X_n \} \right \}$$

The maximum likelihood estimator for $\theta$ can be found by maximizing

$$ \sum_{i = 1}^{n-1} \log( f( X_{i+1} | X_i, \theta)) $$

and we can notice that (assuming the true model has an auto regressive structure with $\theta_0$ as the true parameter)

$$ \frac{1}{n-1} \sum_{i = 1}^{n-1} \log( f( X_{i+1} | X_i, \theta)) \rightarrow E[ f( X | \theta) ] $$

somehow through the ergodic theorem of stationary time series. This last limit is obviously not correct in the stated form, so how is this really done?