Clarification of finding this transition probability matrix

90 Views Asked by At

Let $X_n$ denote the two-state Markov chain with transition probability matrix P= $ \begin{bmatrix} \alpha & 1-\alpha\\ 1-\beta & \beta \end{bmatrix} $ given states 0 and 1. Let $Z_n=(X_{n-1},X_n)$ be a Markov chain over the four states (0,0), (0,1), (1,0), and (1,1). Determine the transition probability matrix.

Here are my thoughts so far: If $Z_n=(X_{n-1},X_n)$, then $Z_{n-1}=(X_{n-2},X_{n-1})$. I believe I'm looking for $P[Z_n=(x_1,y_1)|Z_{n-1}=(x_2,y_2)]=\frac{P[X_{n-1}=x_1, X_n=y_1, X_{n-2}=x_2, X_{n-1}=y_2]}{P[X_{n-2}=x_2, X_{n-1}=y_2]}$=$P(X_n=y_1|X_{n-1}=x_1, X_{n-2}=x_2)$=$P(X_n=y_1|X_{n-1}=x_1)$ by the Markov property.

For example, using the points (0,0)$\rightarrow$(0,0) would be $P[Z_n=(0,0)|Z_{n-1}=(0,0)]=\frac{P[X_{n-1}=0, X_n=0, X_{n-2}=0, X_{n-1}=0]}{P[X_{n-2}=0, X_{n-1}=0]}$=$P(X_n=0|X_{n-1}=0, X_{n-2}=0)$=$P(X_n=0|X_{n-1}=0)$ which corresponds to $p_{00}=\alpha$.

Following this process I obtained P=$\begin{bmatrix} \alpha &0 &\alpha &0\\ 1-\alpha &0 &1-\alpha &0\\ 0 &1-\beta &0 &1-\beta\\ 0 &\beta &0 &\beta\\ \end{bmatrix}$.

$0$'s denote a probability that is not possible (i.e. $X_{n-1}$ cannot be both 0 and 1).

I've seen the answer expressed as the matrix P= $\begin{bmatrix} \alpha &1-\alpha &0 &0\\ 0 &0 &1-\beta &\beta\\ \alpha &1-\alpha &0 &0\\ 0 &0 &1-\beta &\beta\\ \end{bmatrix}$

Based on the logic in when a probability equals zero I really think my transition probability matrix is correct rather than the alternative choice. Any thoughts?

1

There are 1 best solutions below

2
On BEST ANSWER

Your matrix is just the transpose of the answer. It occurs probably because your definition is different from the answer's. For instance, for the transition probability matrix $P$, the row sums equal to 1. It means that $$ \sum_{j} p_{ij} = 1 $$ because it is summing over all possible outcomes of $i \to j$. It does not matter in computing the probability for $(0,0) \to (0,0)$, but it does matter for $(1,0) \to (0,0)$. Examine your calculation and make sure that it is defined as $$ p_{ij} = \mathbb{P}(Z_{n+1}=j|Z_n=i) $$ Yours may be the other way around and result in the transpose.