Solving Markov chain Conditional probability

Question

i am struggling to do the first part which im certain is needed to also complete the other parts aswell, i know i can use the property of conditional probability however i am unsure of how to find for example p(X4=j,X3=2)

Answer 1

Hint : $$ P(X_3 = 2) = \sum_{i=1}^2 P(X_3 = 2 | X_1 = i)P(X_1 = i) $$

Now, for each $i$, $P(X_1 = i) = 0.5$. Furthermore, we have : $$ P(X_3 = 2 | X_1 = i) = \sum_{j = 1}^2 P(X_3 = 2 | X_2 = j , X_1 = i) P(X_2 = j | X_1 = i) \\ = \sum_{j=1}^2 P(X_3 = 2 | X_2 = j) P(X_2 = j | X_1 = i) = \sum_{j=1}^2 p_{i,j}p_{j,2} $$

Whence combining : $$ P(X_3 = 2) = \sum_{i,j=1}^2 p_{i,j}p_{j,2}P(X_1 = i) $$

Which can be calculated. Writing it out explicitly : $$ P(X_3 = 2) = \\ p_{1,1}p_{1,2}P(X_1 = 1) \\ + p_{2,1}p_{1,2}P(X_1 = 2) \\ + p_{1,2}p_{2,2}P(X_1 = 2 ) \\ + p_{2,2}p_{2,2} P(X_1 = 2) $$

Now you know all the quantities above, thence can conclude.

Now for parts $b$ and $c$, use the Bayes' rule : $$ p(X_2 = 1 | X_3 = 2) = \frac{P(X_3 = 2 | X_2 = 1)P(X_2 = 1)}{P(X_3 = 2)} $$

which can be calculated. Similarly the third one.

Third one : $$ P(X_1 = 1 | X_3 = 2) = \frac{P(X_3 = 2 |X_1 = 1)P(X_1 = 1)}{P(X_3 = 2)} $$

The $P(X_3 = 2 | X_1 = 1)$ can be calculated as $p_{1,1}p_{1,2} + p_{1,2}p_{2,2}$ from the formula mentioned in part $1$. The rest are known.