The definition that I have of a Transition Matrix for a Markov Chain is:
A transition matrix is regular if some power of it is positive.
Doesn't this mean though that in theory, you could keep calculating powers forever, because at some point one of the future transition matrices could become positive (assuming it's not already)?
Is there a more exact way checking? I have a transition matrix that I need to show is non-regular. For all the powers I've tried, it contains a 0, but for all I know, that could change if I kept checking forever.
The matrix:
$\begin{bmatrix}0.7 & 0 & 0.3\\0 & 1 & 0 \\ 0.2 & 0 & 0.8\end{bmatrix}$
Solution 1: Denote by $P$ the transition matrix and by $P^n = (P^n(i,j))_{i,j=1,2,3}$ the $n$-th power of $P$. Let $(X_n)_{n \in \mathbb{N}}$ be a Markov chain with transition matrix $P$. It is known that
$$P^n(i,j) = \mathbb{P}^i (X_n=j), \tag{1}$$
i.e. $P^n(i,j)$ is the probability that the Markov chain starts at $x=i$ and moves to $x=j$ in $n$ steps. However, since $\mathbb{P}^2(X_1 = 2) = 1$, it follows (e.g. from the Markov property) that
$$\mathbb{P}^2(X_n = 2)=1$$
for all $n \in \mathbb{N}$. (Hint: Draw a graph of the chain!) This means that the Markov chain stays at $x=2$ whenever it starts at $x=2$. Therefore, $(1)$ is not satisfied for any $n \in \mathbb{N}$ for $i=2$, $j=1$.
Solution 2: Given two matrices $P,Q$ of the form
$$ P = \begin{pmatrix} p_{11} & 0 & p_{13} \\ 0 & 1 & 0 \\ p_{31} & 0 & p_{33} \end{pmatrix} \qquad \quad Q = \begin{pmatrix} q_{11} & 0 & q_{13} \\ 0 & 1 & 0 \\ q_{31} & 0 & q_{33} \end{pmatrix},$$
it is not difficult to see that $P \cdot Q$ is again of this form, i.e.
$$R := P \cdot Q = \begin{pmatrix} r_{11} & 0 & r_{13} \\ 0 & 1 & 0 \\ r_{31} & 0 & r_{33} \end{pmatrix}. \tag{2}$$
Since the transition matrix $P$ is of the form, we find that $P^2$ is also of this form and, by iteration, that $P^n$ is of the form $(2)$ for any $n \in \mathbb{N}$. Consequently, the matrix is not regular.