Markov chain problem, Help!

Question

I am stuck on this question for a long time Question:

Consider 4 balls, labelled from 1 to 4 and distributed amongst two urns (Urn 1 and Urn 2). At each time $n>1$, a number from 1 to 4 is chosen at random. The ball with that label is then transferred from one urn to the other. Each ball is equally likely to be chosen at any time. Let $Xn$ denote the number of balls in Urn 1 at time n.

(i) Draw the steady state diagram and construct the transition matrix for the process $Xn$.

i have drawn the steady state diagram and get the transition matrix \begin{bmatrix}0&1&0&0&0\\1/4&0&3/4&0&0\\0&1/2&0&1/2&0\\0&0&3/4&0&1/4\\0&0&0&1&0\end{bmatrix}

(ii). Using MATLAB, or otherwise, investigate the long-run behaviour of the probability state vector $P^{n}$ at time n. [Display outputs of $P^{n}$; perhaps via a plot of each component versus time n.] Does your answer depend on the initial state? Find the eigenvalues of T (a numerical estimate will suffice)

I dont know how to start part 2, i know $P^{n}$ = $P^{0}T^{n}$

How can i find $p^{0}$?

Answer 1

You should find that, for every initial distribution $P^0$, the distribution $P^n$ at time $n$ is some linear combination $P^n=pR_n+(1-p)S_n$ where:

• $p=P^0(0)+P^0(2)+P^0(4)$
• $R_n$ and $S_n$ are probability distributions
• $R_{2n}$ and $S_{2n+1}$ have support $\{0,2,4\}$
• $S_{2n}$ and $R_{2n+1}$ have support $\{1,3\}$
• $R_{2n}$ and $S_{2n+1}$ converge to $\frac18\cdot(1,0,6,0,1)^T$
• $S_{2n}$ and $R_{2n+1}$ converge to $\frac12\cdot(0,1,0,1,0)^T$

Thus, $P^n$ converges if and only if $p=\frac12$, in which case $$ \lim_{n\to\infty}P^n=\tfrac1{16}\cdot(1,4,6,4,1)^T. $$