Calculate the expected value for this markov chain

Question

Harry's restaurant changes year after year between the states $0$ (bankrupt), $1$ (almost bankrupt) and $2$ (solvent).

The transition matrix is $P= \begin{pmatrix} 1 & 0 & 0 \\ 1/2 & 1/4 & 1/4 \\ 1/2 & 1/4 & 1/4 \end{pmatrix}$

Calculate the expected value for the amount of years till state $0$ is reached, if we started from state $2$.

I took this question from an exam and try to solve it but I'm not sure how to do this correct? I'm a bit confused we need to work with expected value to calculate the required steps / years to get from state $2$ to state $0$. At least that's how I understood this so far.

It all sounds like I need to solve some recursive relations. Let $h(k)$ be the expected number of steps / years in this example until we reach the state $0$ when you are in state $2$. So we have that $$h(2)=0$$

because when you are in state $2$, you need $0$ steps / years to reach $2$. Then for $k=1$

$$h(1) = 1+0.25h(1)+0.25h(2)+0.5h(0)$$

because when you are in state $1$ you will need a step ($+1$) so you will reach with probability $0.25$ state $1$ again and with probability $0.25$ state $2$ and with probability $0.5$ state $0$.

Similarly we do this for $h(0):$

$$h(0) = 1+1h(0)$$

But from here I don't really know how to continue to get the system and calculate the expected number pf steps with that? : /

Answer 1

Let $h(k)$ be the expected time to reach state $0$ if we started from state $\color{blue}k$.

Then $h(0)=0$.

And if we start with state $1$, with probability $\frac12$ we reach state $0$, with probability $\frac14$ we reach state $1$, and with probability $\frac14$ we reach state $2$.

Hence $$h(1)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$

Similarly,

$$h(2)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$

Substituting $h(0)=0$, we have

$$h(1)=1+\frac14h(1)+\frac14h(2)\tag{1}$$

$$h(2)=1+\frac14h(1)+\frac14h(2)\tag{2}$$

Subtracting both equation we have $$h(1)=h(2)\tag{3}$$

Use equation $(3)$ and $(2)$ to solve for $h(2)$.

Answer 2

This is a Markov chain with an absorbing state. Denote by $Q$ the non-absorbing block of the transition matrix. The fundamental matrix is $N= (I-Q)^{-1} = $

$$ \begin{pmatrix} \frac{3}{4} & -\frac{1}{4} \\ -\frac{1}{4} & \frac{3}{4} \\ \end{pmatrix} ^{-1} = \frac{1}{2} \begin{pmatrix} 1 & 3\\ 3 & 1\\ \end{pmatrix} $$

This matrix's entry $(i, j)$ is the expected value of visits at $i$ before being absorbed if the chain starts at $j$ (or the other way around, I don't remember, but luckily it doesn't matter in this case the matrix is symmetric). So the answer is

$$\frac{1}{2}(3+1) = 2.$$

---

EDIT

The meanings of the entries of $N$ are actually other way around (since the transition matrix is given such that "rows sum to $1$"). Here's a reference (page 15 of the pdf (or 419 as the page number)).