Exercise :
Let $\{ X_n \}_{n \in \mathbb{N}_0}$ be a Markov Chain over the space $\mathbb{X} = \{ 1,2,3 \}$ with the following transition matrix : $$P = \begin{pmatrix} 0 & 1/2 & 1/2 \\ 1/3 & 1/3 & 1/3 \\ p & 2/3 -p & 1/3 \end{pmatrix}$$ Calculate the probability $\mathbb{P}[X_n=1|X_0=1]$ for all $n \in \mathbb{N}$ for the cases of a) $p=0$, b) $p =1/6$, c) $p=2/3$. How would you calculate $\lim_{n\to \infty} P^n$ without many operations ?
Attempt :
First of all, I start by finding the eigenvalues and eigenvectors for $P$, which are :
$$\det(P-\lambda I) = 0 \Rightarrow \begin{vmatrix} 0 & 1/2 & 1/2 \\ 1/3 & 1/3 & 1/3 \\ p & 2/3 -p& 1/3 \end{vmatrix} = 0 \Leftrightarrow \dots \Leftrightarrow \lambda = \begin{cases} 1 \\ \frac{1}{6}\big(\pm\sqrt{6p-1}-1\big)\end{cases}$$
The problem I have though, is that for example, the case $p=0$ yields complex eigenvalues, which I don't know how to handle in terms of probabilities to continue on. Also, in the case $p=1/6$ we have a Jordan case, which still leaves us with a hard case for $P^n$. Any tips on how to calculate the given probability ?
I would guess its something like this:
$x_{n}$ = $A^{n}$ * $x_{0}$
$x_{n}$ = ($P^{-1} L^{n} P$) * $x_{0}$
Where P is the matrix made of all the eigenvectors of A, L is the matrix with the corresponding eigenvalues, and $x_{0}$ is the unit vector [1, 0, 0].
Since you only care about the probability Pr($X_{n}$ = 1 | $X_{0}$ = 1), you only care about the element $A^{n}_{0,0}$. Since you have complex eigenvectors, your final result should be real positive number.