Find the limit of $P_s(n)^k$ in a markov chain

27 Views Asked by At

I have a markov chain like this

$$ \begin{bmatrix} P_{s}(n) \\ P_{f}(n) \end{bmatrix} = \begin{bmatrix} a & b \\ 1-a & 1-b \end{bmatrix} \begin{bmatrix} P_{s}(n-1) \\ P_{f}(n-1) \end{bmatrix} $$

where

$$ \begin{bmatrix} P_{s}(0) \\ P_{f}(0) \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} $$

and I need to calculate $$\lim_{n\to\infty} P_{s}(n)^k,$$ for some constant $k$, if there is any limit.

I tried to manually calculate $P_{s}(n)^k$, in which $k=6, a=0.83, b=0.625$, and I noticed that the curve seems to have a limit (I'm not sure though).

1

There are 1 best solutions below

3
On BEST ANSWER

Hint: Do not think about the $k$ we will try to calculate $\lim_{n\to \infty} P_s(n)$ first and then take it to the $k^\text{th}$ power.

We have $p_n = Tp_{n-1}$ if we have convergence we should get $p_\infty = Tp_\infty$ or

$$\begin{bmatrix} P_s(\infty)\\P_f(\infty)\end{bmatrix}=\begin{bmatrix} a & b\\1-a & 1-b\end{bmatrix}\begin{bmatrix} P_s(\infty)\\P_f(\infty)\end{bmatrix}.$$

This equation is the eigenvalue equation for the eigenvalue $1$. Solve this equation for $$\begin{bmatrix} P_s(\infty)\\P_f(\infty)\end{bmatrix}.$$

Can you solve it from here?