Evaluate $\mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)$ where $\{X_n\}_{n\in \mathbb{N}_0}$ is a Markov Chain with a given transition matrix $P$.

35 Views Asked by At

Suppose we are given a time homogeneous Markov chain $\{X_n\}_{n\geq 0}$ with state space $\mathcal{X} = \{1,2,3\}$ and transition matrix

$$P = \begin{pmatrix}0 & 1 & 0 \\ 0 & 2/3 & 1/3 \\ p & 1-p & 0 \end{pmatrix}.$$

The goal is to calculate the probability $\mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)$ in each of the cases where (i) $p = 1/16$, (ii) $p=1/6$ and (iii) $p=1/12$.

Case $p = 1/16$

I know there is a nice methodology that approaches this problem via the diagonilization of the matrix $P$. For example, in the first case ($p =1/16)$ we may find that

$$\tag{1} Q^{-1}PQ = \Delta = \begin{pmatrix}1 & 0 & 0 \\ 0 & -1/4 & 0 \\ 0 & 0 & -1/12\end{pmatrix},$$

where

$$Q = \begin{pmatrix}1 & -4 & -12 \\ 1 & 1 & 1\\ 1 & -33/12 & -81/36\end{pmatrix},$$

is the matrix that has eigenvectors of $P$ as columns. Solving for $P$ in $(1)$ and taking $n$-th powers we get

$$\tag{2}P^n = Q\cdot \begin{pmatrix}1 & 0 & 0 \\ 0 & \left(-\frac{1}{4}\right)^n & 0 \\ 0 & 0 & \left(-\frac{1}{12}\right)^n \end{pmatrix}\cdot Q^{-1}.$$

And therefore, we conclude that

\begin{align} \mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)&= P^{n}_{(1,1)}=\left(1,\, \frac{(-1)^{n+1}}{4^{n-1}},\,\frac{(-1)^{n+1}}{12^{n-1}}\right)\cdot Q^{-1}_{(1)}\\ &=\left(1,\, \frac{(-1)^{n+1}}{4^{n-1}},\,\frac{(-1)^{n+1}}{12^{n-1}}\right)\cdot \left(1/65,\, 1/10,\, -3/26\right)^T\\ &= \frac{1}{65}+\frac{1}{10}\cdot \frac{(-1)^{n+1}}{4^{n-1}} - \frac{3}{26}\cdot \frac{(-1)^{n+1}}{12^{n-1}} \end{align},

where by $Q^{-1}_{(1)}$ I denote the first column of the matrix $Q^{-1}.$

Any approaches for $p = 1/6$ or $p = 1/12$?

Therefore, in the case where $p = 1/16$ there is a closed form of the probability $\mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)$. However, in the other two cases where $p = 1/6$ or $p=1/12$ the matrix $P$ is not diagonizable and we cannot proceed in the same manner. Is there any way to achieve a closed form of $\mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)$ in the other two cases? I tried to used the fact that $x = (1,1,1)^T$ is an eigenvector and through that to calculate the element in the $(1,1)$ position of the matrix $P^n$ but it gets rather complicated. I might missing something and I was wondering if there is any other way.