General Two-State Markov Chain: $P(X_{n}=1)=\frac{b}{a+b}+(1-a-b)^n \big(P(X_0=1)-\frac{b}{a+b}\big)$

Question

Consider a general chain with the state space $S=\{1,2\}$ and write the transition probability as $$\begin{pmatrix} 1-a&a\\ b&1-b\end{pmatrix}$$ Use the Markov property to show that $$P(X_{n}=1)=\dfrac{b}{a+b}+(1-a-b)^n \left(P(X_0=1)-\dfrac{b}{a+b}\right)$$.

So, I started off with \begin{align*} P(X_{n+1}=1)&=P(X_{n+1}=1|X_n=1)P(X_n=1)+P(X_{n+1}=1|X_n=2)P(X_n=2)\\ &=(1-a)P(X_n=1)+bP(X_n=2)\\ &=(1-a)P(X_n=1)+b(1-P(X_n=1))\\ &=(1-a-b)P(X_n=1)+b \end{align*} Then, I manipulated the form to show that \begin{align*} P(X_{n+1}=1)-\dfrac{b}{a+b}&=(1-a-b)P(X_n=1)-\dfrac{(1-a-b)b}{a+b}\\ P(X_{n+1}=1)&=(1-a-b)P(X_n=1)+b \end{align*} Any attempt I made to show that the original statement is true from the result that I got led me nowhere.

Answer 1

From

$$P(X_{n+1}=1)=(1-a-b)P(X_n=1)+b,$$

You can conclude that

\begin{align} P(X_{n+1}=1)&=(1-a-b)\left[(1-a-b)P(X_{n-1}=1)+b\right]+b\\ &=(1-a-b)^2P(X_{n-1}=1)+(1-a-b)b+b \end{align}

Repeating the procedure, we have

\begin{align} P(X_{n}=1)&=(1-a-b)^nP(X_0=1)+b \sum_{i=0}^{n-1}(1-a-b)^i\\ &=(1-a-b)^nP(X_0=1)+b \left[\frac{1-(1-a-b)^n}{1-(1-a-b)}\right]\\ &=(1-a-b)^nP(X_0=1)+b \left[\frac{1-(1-a-b)^n}{a+b}\right]\\ &=\frac{b}{a+b}+(1-a-b)^n\left[P(X_0=1)-\frac{b}{a+b}\right] \end{align}

Answer 2

Hint. From the relation $$ u_{n+1}=(1-a-b)u_n+b,\quad n\geq0, \tag1 $$ let's find a fixed real number $\alpha$ such that $$ \left(u_{n+1}-\alpha\right)=(1-a-b)\left(u_n-\alpha\right). \tag2 $$

Inserting $(1)$ in $(2)$ and expanding the right hand side gives $$ \color{red}{(1-a-b)u_n}+b-\alpha=\color{red}{(1-a-b)u_n}-\alpha(1-a-b) $$ that is, after symplyfying,

$$ \alpha=\frac{b}{a+b}, \qquad a+b \neq 0.\tag3 $$

Now from $(2)$, we get that as $u_n-\alpha$ is a geometric sequence:

$$ u_n-\alpha=(1-a-b)^n\left(u_0-\alpha\right) \tag4 $$

equivalently, using $(3)$,

$$ u_n=\dfrac{b}{a+b}+(1-a-b)^n \left(u_0-\dfrac{b}{a+b}\right)\tag5 $$

as wanted.