My professor left us with the following question to work on for ourselves.
Jay has a future and is going long. It will expire in 39 days. You know beforehand it will go up 2/3 and down 1/3 of the days by 1$.
When should you exit? What is the expected val of the optimal strategy?
Expected value of the future is 13 or +1/3 per day, but how does one optimize for a better expected value/exit strategy knowing only this?
This is an exercise in dynamic programming. I assume that all $\binom{39}{13}$ combinations of "up" and "down" days are equally likely.
Let $V(t, r)$ denote the expected value of the optimal strategy at time $t, 0 \leq t \leq 39$ when there are $r$ "up" days remaining, $0 \leq r \leq 26$. The function $V$ is only defined for valid pairs $(t,r)$, and not all pairs are valid. It is immediate that $V(39, 0) = 13$, and we wish to evaluate $V(0, 26)$.
Let $T = 39$. At time $t = T-1$, either $r = 0$ or $r = 1$. If $r = 0$, there is no advantage to staying, so we exit and win $V(T-1,0) = 14$. If $r = 1$, we stay and win $V(T-1,1) = 13$.
At time $t = T-2$, either $r = 0$, $r = 1$ or $r = 2$. If $r = 0$, we exit and win $V(T-2, 0) = 14$. If $r = 2$, we stay and win $V(T-2, 2) = 13$. If $r = 1$, then one of the remaining days is "up" and one of them is "down", with equal probability. Then $$V(T-2, 1) = \frac{1}{2} V(T-1, 0) + \frac{1}{2} V(T-1, 1) = 13.5.$$
One continues recursively, namely by $$V(t, r) = \max\left(\frac{r}{T-t} V(t+1, r-1) + \left(1-\frac{r}{T-t}\right)V(t+1,r), 2(26-r)-t\right).$$ The above equation is a particular instance of the Bellman equation. The fact that there are two quantities in the maximum reflects the binary nature of our choices: stay or exit. The quantity $2(26-r)-t$ is how much we win if we exit at $(t,r)$, and is simply a rewriting of $$\#\{ \text{up days by time } t\}-\#\{ \text{down days by time } t\} = (26 - r) - (t - (26 -r)).$$
Evaluating $V(t,r)$ is sufficient to determine an optimal strategy in this case, since we can simply evaluate both arguments of the maximum to determine whether to leave or exit. When I implement this in Python with memoization (caching already computed values of $V(t,r)$), I obtain $V(0,0) \approx 13.61$.