Markov chains - expected hitting times

123 Views Asked by At

Consider the following markov chain with state space $\{1,2,3,4\}$ and transition matrix

$\begin{pmatrix}1/3&&1/3&&0&&1/3 \\ 1/4&&1/4&&1/4&&1/4 \\ 0&&0&&1/2&&1/2 \\ 0&&0&&0&&1\end{pmatrix}$

and initial distribution $\lambda=(1/2,1/2,0,0)$.

Let $T=\inf\{n\ge0:X_n \in \{1,4\}\}$.

And let $k_i:=\mathbb E_i[T]$ be the expected hitting time wrt $\mathbb P_i$.

How can I calculate the variance of $T$ wrt $\mathbb P_3$?

I know that $Var(T)=\mathbb E_{\mathbb P_3}[T^2]-(\mathbb E_{\mathbb P_3}[T])^2$

How can I determine the first summand $\mathbb E_{\mathbb P_3}[T^2]$?