Given the following jump chain:
\begin{pmatrix} 0 & 1 & 0 & 0 & 0 & 0\\ 1 & 0 & 0 & 0 & 0 & 0\\ \frac{1}{4} & 0 & 0 & \frac{1}{4} & \frac{1}{2} & 0\\ 0 & 0 & 0 & 0 & 0 & 1\\ 0 & 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2}\\ 0 & 0 & 0 & 1 & 0 & 0\\ \end{pmatrix}
And that $q_1=q_3=q_5=12$ and $q_2=q_4=q_6=6$. Given as well that $X_0=3$ what are:
$\lim_{t\to\infty} P(X_t=2|X_0=3)$
the distribution of the number of jumps to state 3
I have found the Q matrix and computed part 1 as follows but I am not sure if it is correct and I do not understand how to do part 2.
Q-matrix:
\begin{pmatrix} -12 & 12 & 0 & 0 & 0 & 0\\ 6 & -6 & 0 & 0 & 0 & 0\\ 3 & 0 & -12 & 3 & 6 & 0\\ 0 & 0 & 0 & -6 & 0 & 6\\ 0 & 0 & 6 & 0 & -12 & 6\\ 0 & 0 & 0 & 6 & 0 & -6\\ \end{pmatrix}
$\lim_{t\to\infty} P(X_t=2|X_0=3)= h_3^{\{1,2\}}\lim_{t\to\infty} P(X_t=2|X_0\in \{1,2\}) = 1 h_3^{\{1,2\}} = \frac{1}{3}$ \
where $h_3^{\{1,2\}}$ is the probability to $\{1,2\}$ starting from 3.
I highly recommend that you draw a state transition diagram to solve this problem. If you do this you'll clearly see that $\{1,2\}$ and $\{4,6\}$ are absorbing and $P(X_{2n-1}=2|X_0=3)=0$ for any $n\in \mathbb{N}$ i.e. the Markov chain can only reach state $2$ on an even$-$numbered step originating at state 3. In fact, we have for large $n$ that $$P(X_{2n}=2|X_0=3)\approx P\Big(\text{Absorbed in \{1,2\}}\Big|X_0=3\Big)$$ To calculate the probability on the right hand side, set $$a_i=P\Big(\text{Absorbed in \{1,2\}}\Big|X_0=i\Big)$$ for $i\in\{1,\ldots,6\}$. Clearly $a_1=a_2=1$ while $a_4=a_6=0$. Using the total law of probability with recursion we have $$a_3=p_{31}a_1+p_{34}a_4+p_{35}a_5=\frac{1}{4}+\frac{1}{2}a_5=\frac{1}{2} \\ a_5=p_{56}a_6+p_{53}a_3=\frac{1}{2}a_3$$ Solving this system yields $a_3=\frac{1}{3}$ and $a_5=\frac{1}{6}$. To summarize, $$\lim_{n\rightarrow \infty}P(X_{2n-1}=2|X_0=3)=0 \\ \lim_{n\rightarrow \infty}P(X_{2n}=2|X_0=3)=\frac{1}{3}$$ For the second part, let $X$ represent the number of jumps to state #$3$. Clearly $X$ is supported on the set $\{0,1,2,\ldots\}$. Notice $$p_X(0)=p_{31}+p_{34}+p_{35}p_{56}=\frac{3}{4}$$ and for $k \geq 1$ we have $$p_{X}(k)=\frac{3}{4}\times\Big(\frac{1}{2}\Big)^{2k}$$