Let $X_0, X_1, ...$ be a Markov Chain with transition matrix \begin{bmatrix} 0 &1/2 & 1/2\\ 1&0&0\\ 1/3&1/3&1/3 \end{bmatrix}
and initial distribution $α = (1/2, 0, 1/2)$.
Find the following:
(a) $P(X_2 = 1 | X_1 = 3)$
(b) $P(X_1 = 3, X_2 = 1)$
(c) $P(X_1 = 3 | X_2 = 1)$
(d) $P(X_9 = 1 | X_1 = 3, X_4 = 1, X_7 = 2)$
My solution:
(a) $P(X_2=1|X_1=3) = \frac{1}{3}$.
(b)
$P(X_2=1, X_1=3) \Rightarrow P(X_2=1|X_1=3) \cdot P(X_1=3)$
Now, $ \alpha P = \begin{bmatrix} \frac{1}{2} & 0 & \frac{1}{2} \end{bmatrix}\cdot \begin{bmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ 1 & 0 & 0 \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \end{bmatrix} \Rightarrow \begin{bmatrix} \frac{1}{6} & \frac{5}{12} & \frac{5}{12} \end{bmatrix} $
So, $ P(X_2=1, X_1=3) = \frac{1}{3} \cdot \frac{5}{12} = \frac{5}{36} $
(c)
$P(X_1=3|X_2=1) = \frac{P(X_2=1|X_1=3) \cdot P(X_1=3)}{P(X_2=1)}$
Now, $P(X_2=1|X_1=3) = \frac{1}{3}$
$\alpha P = \begin{bmatrix} \frac{1}{6} & \frac{5}{12} & \frac{5}{12} \end{bmatrix}$. So, $P(X_1=3) = \frac{5}{12}$.
$\alpha P^2 = \begin{bmatrix} \frac{1}{6} & \frac{5}{12} & \frac{5}{12} \end{bmatrix}\cdot \begin{bmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ 1 & 0 & 0 \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \end{bmatrix} = \begin{bmatrix} \frac{5}{9} & \frac{2}{9} & \frac{2}{9} \end{bmatrix}$. So, $P(X_2=1) = \frac{5}{9}$.
Hence, $P(X_1=3|X_2=1) = \frac{\frac{1}{3} \cdot \frac{5}{12}}{\frac{5}{9}} = \frac{1}{4}$.
(d)
$P^2 = \begin{bmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ 1 & 0 & 0 \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \end{bmatrix} \cdot \begin{bmatrix} 0 & \frac{1}{2} & \frac{1}{2} \\ 1 & 0 & 0 \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \end{bmatrix} = \begin{bmatrix} \frac{2}{3} & \frac{1}{6} & \frac{1}{6} \\ 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{4}{9} & \frac{5}{18} & \frac{5}{18} \\ \end{bmatrix}$.
So, $P(X_9=1|X_1=3,X_4=1,X_7=2) = P(X_9=1|X_7=2) = 0$
Is my calculation correct?
Looks good to me!
You obviously have already mastered conditional probability. And you are using the Markov property in (d) as expected. So, good solution -- that the professor can put on the course webpage :)
If I have to add something, maybe under time constraint (like in an exam), you can take shortcuts by computing only the entries you need. Like in (d), since you only need entry [2,1], just take the dot product of the second row with the first column of the transition matrix to get that particular entry (spending just a fraction of the time).