Suppose I have a transition probabilities
$P_{i, i+1} = p_i$
$P_{i, i - 1}=q_i$
$P_{i, 1}=1-p_i-q_i$
That is, there is probability of going one step forward, one step back, or back to the start. And there is an associated cost to take a step at each state $C_i$. I would like to find
- The expected cost of going from state i to state j.
- The expected steps required to go from state i to state j.
Obviously this is a trivial problem to solve using simulations, but it feels like something for which a theoretical result is possible using Markov chain theory, but it's been a decade since I worked on problems like this in queueing theory.