How to calculate expected Markov transition cost

39 Views Asked by At

Suppose I have a transition probabilities

$P_{i, i+1} = p_i$

$P_{i, i - 1}=q_i$

$P_{i, 1}=1-p_i-q_i$

That is, there is probability of going one step forward, one step back, or back to the start. And there is an associated cost to take a step at each state $C_i$. I would like to find

  1. The expected cost of going from state i to state j.
  2. The expected steps required to go from state i to state j.

Obviously this is a trivial problem to solve using simulations, but it feels like something for which a theoretical result is possible using Markov chain theory, but it's been a decade since I worked on problems like this in queueing theory.