Let $\left(X_{n}\right)_{n}$ be a Markov chain on a finite state space $S$ with transition probabilities $p_{x y}$.

Question

I am struggling with this question and the solutions don't seem to help me much:

Question

Let $\left(X_{n}\right)_{n}$ be a Markov chain on a finite state space $S$ with transition probabilities $p_{x y}$. Suppose $a$ is absorbing and that all other states are transient. Find the minimal nonnegative solution $h$ of the system $$ \left\{\begin{aligned} h(a) &=1 \\ h(x) &=p_{x a}+\sum_{y \in S} p_{x y} h(y), \quad x \neq a \end{aligned}\right. $$ Express $h$ in its simplest form.

Solution

$h(x)=P_{x a}+\sum_{y \in S} P_{x y} h(y)$ where we know $x \neq a$

$$K_{i}^{A}=E_{i}\left(H^{A}\right)=\sum_{n=\infty} n P_{x y}\left(H^{A}=n\right)+\infty\left(H^{A}=\infty\right)$$

$$h_{a}^{i}=P_{a}(h_{\text{heat equation}})$$

$$h_{i}=\frac{(q / p)^{i}-(q / p)^{N}}{1-(2 \mid p)^{N}}$$ $$h_{i}=A+B_{i}$$ $$h_{i}=1-{i / N}$$ $$h_{(x)}=\sum_{j \in A} P_{x q}=\sum_{j \in A} P_{i} j x_{j}$$

I am confused as to how does the heat equation come into play here. As far as I was going about this question it seems like the drunkard's walk? But then the states in that were recurrent so I dont understand the solutions exactly.

Answer 1

Just think about the definition of $h$, $h(a) = P(Ending~up~at~a|being~at~a) = 1$, for the others it's a little more complicated. $$h(x) = h(a)\cdot P(moving~to~a) + \sum_{y \in S,y \ne a}{h(y)\cdot P(moving~to~y)}$$ $h(x)$ is exaclty the probability that you will end up at state a eventually. Since a is the only absorbing state and other states are transient, this is trivially $h(x)=1 (\forall x \in S)$