Three Approaches to a Markov Chain Calculation with Three Different Answers

80 Views Asked by At

I've approached this question three different ways and arrived at three different answers. Something is wrong here...

Consider a Markov chain $X = \{X_n : n \in \mathbb{N}\}$ with probability transition matrix $$ P = \begin{pmatrix} 1/5 & 3/5 & 1/5 \\ 1/2 & 1/2 & 0 \\ 0 & 1/3 & 2/3 \end{pmatrix} $$ and state space $\{0,1,2\}$.

Suppose we want to find the conditional probability $\alpha$ that $X_2 = 2$ given that the chain started at time $0$ in state $0$ and has not yet entered state $1$ by time $2$. That is, (I believe) we want to find $$ \alpha_1 = \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 \ne 1).$$ It seems to me that this should just be $$ \alpha_1 = \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 = 0) + \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 = 2); $$ and since $X$ is a Markov chain, it follows that $$ \alpha_1 = \mathbb{P}(X_2 = 2 | X_1 = 0) + \mathbb{P}(X_2 = 2 | X_1 = 2) = P_{02} + P_{22} = \frac{1}{5} + \frac{2}{3} = \frac{13}{15}. $$ However, another approach suggests that we can compute this probability by first modifying $P$ (and calling the modified matrix $Q$) like so: $$ Q = \begin{pmatrix} 1/5 & 3/5 & 1/5 \\ 0 & 1 & 0 \\ 0 & 1/3 & 2/3 \end{pmatrix}. $$ We then compute $Q^2$: $$ Q^2 = \begin{pmatrix} 1/25 & 59/75 & 13/75 \\ 0 & 1 & 0 \\ 0 & 5/9 & 4/9 \end{pmatrix}. $$ Since the modified process gets absorbed if it hits state $1$, it follows that $$ \alpha_2 = Q^2_{02} = \frac{13}{75} \ne \alpha_1, $$ which contradicts the calculation in Approach #1.

Yet another approach is as follows. We can condition only on the event that $X_0 = 0$ and compute the probability of an intersection of events; i.e., $$ \alpha = \mathbb{P}(X_2 = 2, X_1 \ne 1 | X_0 = 0). $$ Using the "chain rule" for the conditional probabilities yields (I will omit the tedious calculations) $$ \alpha_3 = P_{01}P_{02} + P_{01}P_{12} = \frac{1}{25} + 0 = \frac{1}{25}. $$ So we get yet another contradiction.

My Question:

Are any of these approaches correct? If so, what is wrong with the other two approaches?

UPDATE: After properly rephrasing the question (properly), my thought is that Approach #1 is correct, but that the computation of $\alpha_1$ is incorrect. Also, although Approaches #2 and #3 are incorrect, my thought is that they should yield the same result; i.e., $\alpha_2$ should be equal to $\alpha_3$ (even though my computations suggest otherwise). I'm still trying to get all of this sorted out...

2

There are 2 best solutions below

7
On BEST ANSWER

I don’t understand the third approach, where you left out the details, so I can only speak to the first two.

It seems to me that the problem starts with the imprecise formulation “Suppose we want to find the probability of the chain starting at state $0$ and being in state $2$ on the second step, while avoiding state $1$.”

Here you list three events, all in the same grammatical form, even though some of them are given conditions and others are events whose probability is to be calculated.

You don’t really want to find the probability of the chain starting at state $0$; you take it as given that it started at state $0$. We could write that off as a slight inaccuracy, but from your subsequent calculations it seems that this inaccuracy extends beyond the initial state and also affects your handling of $X_1$.

In your first approach, you’re treating $X_1$ as given; that is, you’re calculating the probability that, given that the chain was in state $0$ at time $0$ and not in state $1$ at time $1$, it will be in state $2$ at time $2$. This is wrong under a literal interpretation of “the probability of ... while avoiding state $1$”, but given that you apparently didn’t mean that literally for state $0$, perhaps you didn’t mean it literally for state $1$, either.

In your second approach, you’re treating $X_1$ as the literal interpretation would suggest, as an unknown random state.

So I think if you think about that sentence that specifies what you want to calculate, and precisely formulate it accordingly, you’ll realize that one of the calculations corresponds to that and the other doesn’t.

By the way, your modification of $P$ is unnecessary, since state $1$ anyway doesn’t transition to state $2$; you get the same result simply from $P^2_{02}$.

0
On

The second approach is correct. If you write out the Bayes Rule you will see that $\mathbb{P}(X_2 = 2 | X_0 = 0, X_1 \ne 1) \neq \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 = 0) + \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 = 2) \wedge \mathbb{P}(X_2 = 2 | X_0 = 0, X_1 \ne 1) \neq \mathbb{P}(X_2 = 2, X_1 \ne 1 | X_0 = 0)$.