I am reading the book Stochastic Processes of a brilliant greek mathematician Nikolaos Skoutaris and in pages 164-165 has the following problem:
Let $X_n, n \in \mathbb{N}_0$ independent r.v following the Bernoulli distribution with parameter $p \in (0,1)$, i.e $$\mathbb{P}(X_n=1)=p ,\mathbb{P}(X_n=0)=1-p.$$ We have a process $\{W_n\}$ where $W_n = \begin{cases} 0, & \text{if}\quad X_n=X_{n-1}=1\\ 1, & \text{otherwise}. \\ \end{cases} $
Is the process a Markov Chain ?
The writer says that the chain $\{W_n\}$ is in the state space \${0,1}$ and is not Markovian because
$$\mathbb{P}(W_n=0|W_{n-1}=1,W_{n-2}=0)= \frac{\mathbb{P}(W_n=0,W_{n-1}=1,W_{n-2}=0)}{\mathbb{P}(W_{n-1}=1,W_{n-2}=0)}=0$$
I understand that he uses the independence of the r.v to calculate the conditional probability function and ends with $\mathbb{P}(W_n=0)$, but how did he find that $\mathbb{P}(W_{n-2})=0$?
He continues by writing that if $W_n=W_{n-2}=0$, then $X_{n-1}=X_{n-2}=1$ and so $W_{n-1}=0$. On the other hand:
\begin{align*} \mathbb{P}(W_n=0|W_{n-1}=1) &= \frac{\mathbb{P}(W_n=0,W_{n-1}=1)}{\mathbb{P}(W_{n-1}=1)} \\ &= \frac{\mathbb{P}(X_n=X_{n-1}=1,X_{n-2}=0)}{1- \mathbb{P}(X_{n-1}=X_{n-2}=1)} \quad\quad \text{(1)}\\ &= \frac{\mathbb{P}(X_n=1) \mathbb{P}(X_{n-1}=1)\mathbb{P}(X_{n-2}=0)}{1- \mathbb{P}(X_{n-1}=1)\mathbb{P}(X_{n-2}=1)}\\ &= \frac{p^2(1-p)}{1-p^2} \\ &= \frac{p^2}{1+p} \\ & \neq 0 \end{align*}
I have two questions :
- How did he find that $P(W_{n-2}=0)$?
- Why can he take the reciprocal in (1)?
Note that the sequence $W_{n-2}=0, W_{n-1}=1$ implies that $X_{n-3}=1, X_{n-2}=1, X_{n-1}=0$. As such, it is impossible that $W_n=0$, because that would require $X_{n}=X_{n-1}=1$. Hence, $\mathbb{P}(W_n=0, W_{n-1}=1, W_{n-2}=0)=0$.
$\mathbb{P}(W_{n-1}=1)=1-\mathbb{P}(W_{n-1}=0)$, and by definition of $W_{n-1}$, you have $\mathbb{P}(W_{n-1}=0)=\mathbb{P}(X_n=X_{n-1}=1)$. So $\mathbb{P}(W_{n-1}=1)= 1- \mathbb{P}(X_n=X_{n-1}=1)$.