Understanding definition of Periodicity of Markov chain

113 Views Asked by At

Consider the following example that is used to understand the definition of periodicity property.

enter image description here

Why does it says that:

starting in state $1$, it is possible for the process to enter state $1$ only at times $2, 4, . . . ,$ so state $1$ has period $2$. The reason is that the player can break even (be neither winning nor losing) only at times $2, 4, . . . ,$ which can be verified by calculating $p_{11}^{(n)}$ for all $n$ and noting that $p_{11}^{(n)} = 0$ for $n$ odd.

I do not understand.

First off isn't $p_{11}^{(n)} = 0$ for all $n$, not just $n$ odd?

By definition $p_{11}^{(n)}=p\{X_{t+n}=1|X_t=1\}$

From the matrix P, $p_{11}^{(n)}=p\{X_{t+n}=1|X_t=1\}=0$, hence the probability will always be zero, no?

1

There are 1 best solutions below

3
On BEST ANSWER

"First off isn't $p_{11}^{(n)} = 0$ for all $n$, not just $n$ odd?"

No. $\ p_{11}^{(2)}= p\left(1-p\right)\ne 0\ $. I suspect you're confusing state $0$ with state $1$. It is true that $\ p_{00}^{(n)}= 1\ $ for all $\ n\ $, so that state is aperiodic.

It is not correct that $\ p\{X_{t+n}=1|X_t=1\}=0\ $. $$ p\{X_{t+n}=1|X_t=1\}=\begin{bmatrix}0&1&0&0 \end{bmatrix}\boldsymbol{\mathrm{P}}^n\begin{bmatrix}0\\1\\0\\0 \end{bmatrix}\ ,$$ and $$ \boldsymbol{\mathrm{P}}^2=\begin{bmatrix} 1&0&0&0\\ 1-p&p\left(1-p\right)&0&p^2\\ \left(1-p\right)^2&0&p\left(1-p\right)&p\\ 0&0&0&1\end{bmatrix}\ .$$ It is the entry in the second row and second column of $\ \boldsymbol{\mathrm{P}}^n\ $that is $\ p_{11}^{(n)}\ $.