To prove that a process is strictly stationary

Question

I have been given $ \{X_n;n>0\}$ as a sequence of i.i.d random variables and we another sequence $ \{Y_n;n>0\}$ defined such that $$Y_n=X_n+aX_{n-1}$$ where a is a real constant

I need to show that $ \{Y_n;n>0\}$ is a strictly stationary process. Now my usual approach to prove a strictly stationary process is to prove that the all the random variables $Y_n$ are independent but I don't think this is necessarily true in this case since the sum of iid RVs is not necessarily an iid itself.

Any hint on how to proceed?

Answer 1

Obviously the sequence $\{X_n\}_{n\geq1}$ is strictly stationary (independence, as you mention). It is left to verify that the function which maps $\{X_n\}_{n\geq1}$ to $\{Y_n\}_{n\geq1}$ behaves “well” when you shift the argument. I will leave you to discover what “well” means here.

EDIT:

The above was a hint. Since it might not have been entirely clear what I meant I decided to provide a full solution as well.

Firstly, it is not clear how $Y_1$ is defined. If one were to define $Y_1:=X_1$ then $\{Y_n\}_{n\geq1}$ is not strictly stationary (unless $a=0$). To get strict stationarity one has to introduce $X_0$ which is independent of $\{X_n\}_{n\geq1}$ and has the same distribution as $X_1$. Then the definition of $Y_n$ also makes sense for $n=1$. Notation wise, however, this is a bit annoying since the sequences $\{X_n\}_{n\geq0}$ and $\{Y_n\}_{n\geq1}$ start at different index. Instead I think the best fix is to change the definition of $Y_n$ slightly. I will define $Y'_n:=X_{n+1}+aX_n$. Note that $\{Y'_n\}_{n\geq1}\sim\{Y_n\}_{n\geq1}$ so we may prove strict stationarity for $\{Y'_n\}_{n\geq1}$ instead.

For $N\in\mathbb{N}$ let $\theta^N$ be the function which maps a sequence $\{x_n\}_{n\geq1}$ of real numbers to the shifted sequence $\{x_{N+n}\}_{n\geq1}$. By definition $\{Y'_n\}_{n\geq1}$ is strictly stationary if $\theta^N(\{Y'_n\}_{n\geq1})\sim\{Y'_n\}_{n\geq1}$ for all $N\in\mathbb{N}$.

As I mention it is fairly obvious that $\{X_n\}_{n\geq1}$ is strictly stationary because it is an i.i.d. sequence. Since the this is simple to prove I am happy to do it here. To show that $\theta^N(\{X_n\}_{n\geq1})\sim\{X_n\}_{n\geq1}$ for some $N\in\mathbb{N}$ we must prove that all finite dimensional distributions are identical. We take integers $0\leq n_1<n_2<\dotsc<n_m$ for some $m\in\mathbb{B}$ and immediately see that $$ (X_{N+n_1},\dotsc,X_{N+n_m})\sim(X_{n_1},\dotsc,X_{n_m}). $$ Indeed, since the $X_n$'s are i.i.d. the left- and right-hand side both have distribution $P_{X_1}^{\otimes m}$ (product measure of $P_{X_1}$ $m$ times), where $P_{X_1}$ is the distribution of $X_1$.

We will then consider the function $f$ which maps $\{X_n\}_{n\geq1}$ to $\{Y'_n\}_{n\geq1}$. To be precise $$ f(\{x_n\}_{n\geq1})_m=x_{m+1}+ax_{m} $$ for $m\in\mathbb{N}$. This function has an important property, namely that $f\circ\theta^N=\theta^N\circ f$ for all $N\in\mathbb{N}$. Indeed, $$ f(\theta^N(\{x_n\}_{n\geq1}))_m=x_{N+m+1}+ax_{N+m}=\theta^N(f(\{x_n\}_{n\geq1}))_m $$ for all $m\in\mathbb{N}$.

The stationarity of $\{Y'_n\}_{n\geq1}$ is obtained from the observations above. We simply see that $$ \theta^N(\{Y'_n\}_{n\geq1})=\theta^N(f(\{X_n\}_{n\geq1}))=f(\theta^N(\{X_n\}_{n\geq1}))\sim f(\{X_n\}_{n\geq1})=\{Y_n\}_{n\geq1} $$ for any $N\in\mathbb{N}$.

Answer 2

Since the author of the OP requested an explicit statement of the stationarity, let us construct the distribution of the $Y_n$'s. Define the probability density $F(X_n=x)=f(x)$

$$F(Y_n=y)=\int_{\mathbb{R}^2}dx_n dx_{n-1}\delta(x_n+ax_{n-1}-y)f(x_n)f(x_{n-1})=\int_{-\infty}^{\infty}dwf(y-aw)f(w)$$

which shows that for all $N\in \mathbb{N}~~$,$~~F(Y_{n+N}=y)=F(Y_n=y)$.

EDIT: It is correct, as stated in a comment, that the statement above is not enough to prove strict stationarity, but it is a key ingredient thereof. To actually prove the strict stationarity of the sequence, it suffices to show that

$$ F_Y(Y_2=y_2,..., Y_n=y_n)=F_Y(Y_{N+2}=y_2,Y_{N+3}=y_3,..., Y_{N+n}=y_n), ~~\forall ~n, N $$

where the $Y_n, n\geq 2$ are defined as above. The reason why this is sufficient is because strict stationarity for an arbitrary set of times ${t_i}$ can be obtained by considering the joint distribution $F_Y(Y_2..., Y_{\max_i t_i})$ and integrating out the times inbetween that do not belong to the set.

In the following I have set $Y_1:=X_1$, because the law of the set $Y_1,Y_2,..., Y_n$ is easily constructible, but eventually stationarity has to be proven without the contribution of $X_1$. Now we just need to construct the probability measure above, which can be done iteratively:

$$F_Y(Y_1=y_1, ..., Y_n=y_n)=F(Y_n|Y_1,..., Y_{n-1})F_Y(Y_1,..., Y_{n-1})$$

The conditional density can be easily computed by independence and the definition of $Y_n$ and the final result is

$$F_Y(Y_1=y_1, ..., Y_n=y_n)=\prod_{k=1}^nf\left(\sum_{\ell=1}^k y_{\ell}(-a)^{k-\ell} \right)\\=f(y_1)f(y_2-ay_1)...f(y_n-ay_{n-1}+\dots+(-a)^ny_1)$$

and thus

$$F_Y(Y_2=y_2,..., Y_n=y_n)=\int dq f(q)f(y_2-aq)...f(y_n-ay_{n-1}+\dots+(-a)^n q)$$

We can now prove the asserted statement. First note that for $N=1$ the statement is true since

$$F_Y(Y_3=y_3,..., Y_{n+1}=y_{n+1})\\=\int dy_1 dy_2f(y_1)f(y_2-ay_1)...f(y_{n+1}-ay_{n}+\dots+(-a)^{n}y_1)\\=\int dt_1 dt_2 f(t_1) f(t_2) f(y_3-at_2)... f(y_{n+1}-ay_n...+(-a)^{n-2}y_3+(-a)^{n-1} t_2)\\=\int dt_2 f(t_2) f(y_3-at_2)... f(y_{n+1}-ay_n...+(-a)^{n-2}y_3+(-a)^{n-1} t_2)=F_Y(Y_2=y_3, ..., Y_n=y_{n+1})$$

Now we perform a similar substitution in the integral for any $N$ and show in a similar manner that

$$F_Y(Y_{N+2}=y_{N+2},..., Y_{N+n}=y_{N+n})\\=\int dy_1...dy_{N+1} f(y_1)...f(y_{N+1}-...+(-a)^{N}y_1)...f(y_{N+n}-...+(-a)^ny_N+...+(-a)^{N+n-1}y_1)$$

Which by setting $t_{N+1}=y_{N+1}-ay_N+...+(-a)^N y_1$ and noting that

$$\int dy_1...dy_N f(y_1)f(y_2-ay_1)...f(y_N-...+(-a)^{N-1}y_1)=1$$

shows that

$$F_Y(Y_{N+2}=y_{N+2},..., Y_{N+n}=y_{N+n})=F_Y(Y_{2}=y_{N+2},..., Y_{n}=y_{N+n})$$

which concludes the proof.