You roll a conventional fair die repeatedly. If it shows 1, you must stop, but you may choose to stop at any prior time. Your score is the number shown by the die on the final roll. What stopping strategy yields the greatest expected score? What strategy would you use if your score were the square of the final roll?
Continuing with Exercise (8), suppose now that you lose $c$ points from your score each time you roll the dies. What strategy maximizes the expected final score if $c= 1/3$? What is the best strategy if $c=1$?
I managed to understand the solution for 8. Suppose that I choose the strategy to roll a die until I get 6. Let this strategy be $S_6$. Then, $E(S_6) = 6 P(\text{6 appears before 1}) + 1 P(\text{1 appears before 6}) = 7/2$. Similarly, I can compute $E(S_5) = 4$, $E(S_4) = 4$, and so on. (the authors assume that when the player choose $S_5$ and 6 turns up, the player also stop playing the next round.)
For 9 (the case for $c = 1/3$), my attempt is as follows. $E(\text{Cost of $S_6$}) = \sum_{k=1}^\infty 1/3 k f_T(k)$ where $T$ is the number of tosses required until the game ends. Since the subsequent toss is needed when 2,3,4,5 appears, $f_T(k) = (2/3)^{k-1}$. Thus, I get $E(\text{Cost of $S_6$}) = 3$, so the expected final score of $S_6$ is 7/2 -3 = 1/2. But, the solution says that the expected final score of $S_6$ is $7/2 -3C = 5/2$. Could you explain how the authors get this result?
They are saying that the expected amount of time $S_6$ takes is 3 turns (it comes from 1/(2/6) which is the mean of a geometric RV. So, the expected reward you are going to get is the answer to part 8 minus the expected loss you suffer from the duration (so 3 C). Similarly, for $S_5$, each round is going to last 1/(3/6) = 2 turns on average and so on and so forth)