Distribution of Markov Chain with transition matrix

165 Views Asked by At

An optional challenge assignment: Given a stationary Markov chain $\mathbf X=(X_k)^\infty_{k=1}$ where $X_k$ takes values in {0,1,2}. Let it have a probability transition matrix $P=[P_{ij}]=Pr(X_{k+1}=j|X_k=i)$ as follows:

$$\begin{bmatrix}i|j & 0 & 1 & 2 \\0 & 1/4 & 1/2 & 1/4\\1 & 1/2 & 0 & 1/2\\2 & 1/4 & 1/2 & 1/4\end{bmatrix}$$

Calculate the distribution $\mathbf p$ of $X_1$, where $p_i=Pr(X_1 = i)$ and $\mathbf p=(p_0,p_1,p_2)$.

I'm given three possible $\mathbf p$: $(0,1,2), (1/4,1/2,1/4),$ and $(1/3,1/3,1/3).$

Thus $P=Pr(X_2=j|X_1=i)$ for $i=0,1,2$. I'm tempted to guess, from the options, that $\mathbf p$ cannot be (0,1,2) since probability greater than one is not possible, and that the correct answer is (1/3,1/3,1/3) by taking the sum of each probability and dividing by three. I'm simply not sure how to solve it.

1

There are 1 best solutions below

0
On BEST ANSWER

Let $\bf{p}=\bf{p}^{(1)}$, $\bf{q}=\bf{p}^{(0)}$

In general, $\bf{p}=\bf{q}\, P$. So, as rightly pointed out in a comment, you need the initial probabilities $\bf{q}$ to get, together with $P$, the desired $\bf{p}$.

Let's see:

$p_0= \frac{1}{4}q_0+\frac{1}{2}q_1+\frac{1}{4}q_2$

$p_1= \frac{1}{2}q_0+\frac{1}{2}q_2$

$p_2= \frac{1}{4}q_0+\frac{1}{2}q_1+\frac{1}{4}q_2$

This implies that, irrespective of the initial $\bf{q}$ , we must have $p_0=p_2$ (both the second and third alternatives fullfill this).

We can simplify the above a little more: $q_1 = 1 - (q_0 + q_2)$ Hence

$p_1= \frac{1}{2}(q_0+q_2)$

$p_0= p_2=\frac{1}{2}- \frac{1}{4}(q_0+q_2)=\frac{1}{2}-\frac{1}{2} p_1$

Hence...