Consider the transition matrix
$ P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} $
for general $2$-state Markov Chain $(0 \le p, q\le 1)$.
- Find the limiting distribution (if it exists) if $p + q \ne 1$.
Using mathematical induction, it is solved using the following proof:
$ p^n = \frac{1}{p+q} \begin{bmatrix}q&p\\q&p\end{bmatrix} + \frac{(1-p-q)^n}{p+q} \begin{bmatrix}p&-p\\-q&q\end{bmatrix} $
This is totally cumbersome.
Can this be solved in any other way like using $\pi (P-I) = 0$ and so on?
Assuming the eigenvalues are distinct, we can find two linearly independent eigenvectors. The result is that we can write $\mathbf{P}$ in the form $P = \mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$ where
$$ \mathbf{Q} = \left[\begin{array}{cc} 1&-p\\ 1&q \end{array} \right], \mathbf{Q}^{-1} = \left[\begin{array}{cc} \frac{q}{p+q}& \frac{p}{p+q}\\ \frac{-1}{p+q}&\frac{1}{p+q} \end{array} \right], \mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&1-p-q \end{array} \right] $$
Note that the columns of $Q$ are right-eigenvectors for $\mathbf{P}$, and the rows of $Q^{-1}$ are left-eigenvectors for $\mathbf{P}$.
The normalized left-eigenvector for eigenvalue 1 is:
$$\left[\begin{array}{c} \frac{q}{p+q} \\ \frac{p}{p+q} \end{array} \right]$$
so this is the stationary distribution. Now, since $0\leq p,q \leq 1$, if one is strictly in $(0, 1)$, the eigenvalue $1-p-q$ has $\left|1-p-q\right| \lt 1$. Therefore
$$ \mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&(1-p-q)^n \end{array} \right] \rightarrow \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right], n \rightarrow \infty $$
And Also
$$ \mathbf{P} = \mathbf{Q}^{-1} \mathbf{D}^n \mathbf{Q} \rightarrow \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ -1/(p+q)& 1/(p+q) \end{array} \right] \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right] \left[\begin{array}{cc} 1&-p\\ 1&p \end{array} \right] = \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ q/(p+q)& p/(p+q) \end{array} \right] = \left[\begin{array}{c} \pi\\ \pi \end{array} \right] $$
as $n \rightarrow \infty$. The chain converges to this stationary distribution regardless of initial distribution, provided at least one of $p, q$ is in $(0, 1)$.