Markov Chain transitional probability query.

Question

Say I have the transitional probability matrix P= $\begin{bmatrix}.8 & .2\\.6 & .4\end{bmatrix}$ And the entry (1,1) denotes the probability that I stay in state 0, (1,2) I move from state 0 to state 1, (2,1) I move from state 1 to state 0 and (2,2) stay in state 1. Now suppose I am on day 1 and in state 0, and I want to find the probability that I will remain in state 0 over day 2, 3 and 4. Am I right in thinking that I compute $P^2 \ entry (1,1) + P^3 \ entry (1,1)+P^4 \ entry (1,1). $ And for when I need to find the probability of being in state 0 on day 4, I only compute $P^4 \ entry (1,1)$.

Answer 1

Revising my opinion, based on comment of @Did. I would like explain his answer in detail to make sure you avoid my mistake.

Let $X_t$ be the position of your process at time $t$. You start with $X_1 = 0$.

You would like to end up with $X_2 = 0$. The probability of that event is $P_{11}$. Since your Markov chain is time homogeneous, $$\mathbb{P}[X_3 = 0|X_2 = 0] = \mathbb{P}[X_2 = 0|X_1 = 0] = P_{11}.$$ So, to go from $X_1 = 0$ to $X_2 = 0$, it's also a transition with probability $P_{11}$.

Now, what is the probability of tracing that path entirely? $$ \begin{split} \mathbb{P}[X_3 = 0,X_2=0|X_1 = 0] &= \mathbb{P}[X_3 = 0|X_2 = 0,X_1 = 0] \cdot \mathbb{P}[X_2 = 0|X_1 = 0] \\ &= \mathbb{P}[X_3 = 0|X_2 = 0] \cdot \mathbb{P}[X_2 = 0|X_1 = 0] \\ &= \left(P_{11}\right)^2. \end{split} $$

Along similar lines, you can show $$ \mathbb{P}[X_n = 0,X_{n-1}=0, \ldots, X_2=0|X_1 = 0] = \left(P_{11}\right)^{n-1}. $$