Regarding a Markov chain example state space $\{0,1\}$

Question

I have trouble formulating a question.

The set up is $(X_n)$ is a Markov chain with the state space $\mathcal{S} = \{0,1\}$.

We know $X_0 =1$ and $X_2=1$ and the transition probability matrix, $p$.

We want to find $P(X_1=1)$.

My first guess is that it would be $p(1,1)$ but it seems reasonable that the knowledge that $X_2=1$ should have effect on the probability in question.

How should I approach this problem?

Thank you in advance!

Answer 1

My answer assumes you actually want to compute $P(X_1=1 \mid X_0=1, X_2=1)$. If you really wanted $P(X_1=1)$, see the answer by @probablyme.

---

By the definition of conditional probability, $$P(X_1=1 \mid X_0=1, X_2=1) = \frac{P(X_0=1, X_1=1, X_2=1)}{P(X_0=1,X_2=1)} = \frac{P(X_0=1, X_1=1, X_2=1)}{P(X_0=1,X_1=0,X_2=1)+P(X_0=1,X_1=1,X_2=1)}$$

By the Markov assumption, $$P(X_0=1,X_1=x, X_2=1) = P(X_0=1) \cdot p(1,x) \cdot p(x,1).$$

Plugging this in above gives $\frac{p(1,1) \cdot p(1,1)}{p(1,0) \cdot p(0,1) + p(1,1) \cdot p(1,1)}$

Answer 2

$p(1,1)$ I'm assuming this notation means probability $P(X_1 = 1|X_0 = 1)$. So, not quite. You're looking for $$P(X_1=1) = P(X_0 = 0)P(X_1=1|X_0 = 0)+P(X_0= 1)P(X_1= 1|X_0=1),$$ by the law of total probability.