I read in a course of linear time series the following proposition without demonstration:
Let $ (X_t)_{t \in \mathbb{Z}}$ be a process that is weak-stationary (i.e. square integrable, with constant mean and $ \forall (t,h) \in \mathbb{Z}^2 ~ ~Cov(X_t, X_{t-h}) = \gamma_X(h)$ ) Then :
$$ \textbf{EL}[X_t|\text{Span}(1, X_{t-1}, …, X_{t-p})] \underset{p \rightarrow +\infty}{\xrightarrow{~~~L^2~~~}} \textbf{EL}[X_t|\mathcal{H}_X(t-1)] $$
With
• $ \textbf{EL}[X_t|\mathcal{H}_X(t-1)] = \underset{Z \in \mathcal{H}_X(t-1)}{argmin}\{ \mathbb{E} [(X_t - Z)^2]\} $ where $ \mathcal{H}_X(t-1) = \overline{\text{Span}(1,X_{t-1},X_{t-2},…)} $
• $ \textbf{EL}[X_t|\text{Span}(1, X_{t-1}, …, X_{t-p})] = \underset{Z \in ~\text{Span}(1, X_{t-1}, …, X_{t-p})}{argmin}\{ \mathbb{E} [(X_t - Z)^2]\} $
I have tried to demonstrate this, notably by trying to apply the Dominated Convergence Theorem, and by showing that $ Z \in \mathcal{H}_X(t-1) \Rightarrow \exists (a_i)_{i \in \mathbb{N}}, ~Z = a_0 + \sum_{i \geq 1} a_i X_{t-i} $ but unfortunately I can't get anywhere.
Thank you !
For convenience, denote $Z:=\textbf{EL}[X_t|\mathcal{H}_X(t-1)]$ and $Z_p :=\textbf{EL}[X_t|\text{Span}(1, X_{t-1}, …, X_{t-p})] $. We are trying to prove that for any $\varepsilon >0$, there exists $p_0\in \mathbb N^*$ such that $\|Z_p - Z\|_{L^2} $ for all $p\ge p_0$.
Let $\varepsilon >0$. We know that $Z\in\overline{\text{Span}(1,X_{t-1},X_{t-2},…)}$ so by definition there exist $\alpha_0,\alpha_1,\ldots\in\mathbb R$ such that
$$\lim_{p\to\infty} \alpha_0 + \sum_{i=1}^p \alpha_iX_{t-i} = Z \tag1$$ Where the limit is understood in the $L^2$ sense. Denote $\tilde Z_p:= \alpha_0 + \sum_{i=1}^p \alpha_iX_{t-i} $ for all $p$. By $(1)$, there exists an index $p_0$ such that $\|\tilde Z_p - Z\|_{L^2}\le \varepsilon $ for all $p\ge p_0$. Now write for all $p\ge p_0$ $$\begin{align*} \|Z-Z_p\|_{L^2}&\le \underbrace{\|Z-\tilde Z_p\|_{L^2}}_{(A)} + \underbrace{\|\tilde Z_p-Z_p\|_{L^2}}_{(B)} \end{align*} $$
Observe that $(A)$ is bounded by $\varepsilon$ by construction while for $(B)$, since $\tilde Z_p\in \text{Span}(1, X_{t-1}, …, X_{t-p})$ we can write $$\begin{align*} \left\|\tilde Z_p-Z_p\right\|_{L^2}&=\left\|\textbf{EL}[\tilde Z_p|\text{Span}(1, X_{t-1}, …, X_{t-p})]-\textbf{EL}[Z|\text{Span}(1, X_{t-1}, …, X_{t-p})]\right\|_{L^2}\\ &=\left\|\textbf{EL}[\tilde Z_p - Z\mid\text{Span}(1, X_{t-1}, …, X_{t-p})]\right\|_{L^2}\\ &\le \|\tilde Z_p - Z\|_{L^2} \le \varepsilon\ \forall p\ge p_0\end{align*}$$
Where we used linearity and triangle inequality for conditional expectation in the last inequality.
It thus follows (up to relabeling $\varepsilon'\equiv 2\varepsilon$) that $\|Z_p-Z\|_{L^2}\le \varepsilon' $ for all $p\ge p_0$, as desired.