A time-dependent state infinite Markov chain optimal stopping time

43 Views Asked by At

This is a variation on this question.

$f$ and $g$ are two (positive) functions on the set of natural numbers $\mathbf N$. You play a multistep game with a die with equal probability of generating any of $1,2,\cdots, n$ at each roll. At the end of $t$'th roll, you may choose to stop the game and walk away with the dollar amount that is $f(t)$ multiple the number generated by the last roll of the die, or choose to pay \$$g(t)$ to continue the game. The game can last indefinitely. What is the amount of money you expect to collect from this game? Is there a "closed expression" for the solution?

The original version of this problem is a Markov chain with time-independent states which leads to a simple solution. The time-dependency of the current version seems to break this symmetry. Is there still a simple solution?