A auto-machine pick up a ball uniformly randomly from a box filled with green balls and red balls in every minute.
When the # of green balls or red balls = 0, the auto-machine do not pick.
Otherwise, if the auto-machine picked a red ball, it will replace one green ball with a red ball .
if the auto-machine picked a green ball, it will replace one red ball with a green ball.
Suppose the total # of balls is fixed.
At each odd minute, you can throw away any non-negative number of red balls, $\textbf{the goal is to maximize the expected number of green in the end} $ (when either red ball count or green ball count goes down to zero). Each step includes: your change, and then change by the auto-machine.
Consider the plan $\bf{A}$: Every time you find $r \ge g>0$, $r=$ the # of red balls and $g=$ # of green balls, you bring the red ball count to $g-1$ by throwing away $r-g+1$ red balls.
Let $W(r,g)$ be the expected # of green balls in the end, following Plan $\bf{A}$ starting with $r$ red balls and $g$ green balls at start.
(a) Show that $W(0,g)=b, W(r,0)=0, W(r,g)=W(r-1,g)$ for $r\ge g>0$ and
$W(r,g) = \frac{r}{r+g}W(r+1,g-1)+\frac{g}{r+g}W(r-1,g+1) \text{ for }0<r<g.$
(b) Prove that $W(R_{n}^{A},G_{n}^{A})$ be martingale w.r.t. some filtration where $R_{n},G_{n}$ are the red and green ball count after the $n$-th change by the auto-machine and when following plan $\bf{A}$
(c) Assuming that (no need to prove it), for all $r,g>0$ \begin{align*} W(r,g)\ge W(r-1,g) \text{ and } W(r,g) \ge \frac{r}{r+g}V(r+1,g-1)+\frac{g}{r+g}V(r-1,g+1), \end{align*} prove that $W(R_{n}^{B},G_{n}^{B})$ is a non-neg supermartingale for any other plan $\bf{B}$.
(d) Show that the plan $\bf{A}$ is the best plan for maximizing the expected # of green balls in the end.
Is there any hint for solving this question? Or is there any reference for this question? Thank you!