The following is a dynamic programming problem where I am having trouble writing the Bellman equation:
My puzzle is that since there is a certain period that we must get a house, the "usual" bellman equation in the form of $V(t,p)=\max_{accept, reject} \{\frac{v}{1-\beta}-p,u+\beta E[V(t+1,p')]\}$ seems to fail to capture the feature that we have to get the house by time $T$.
I have also tried to study the problem by backward induction, like when $t= T-1$, $V(t,p)=\max_{accept, reject} \{\frac{v}{1-\beta}-p,u+\beta E[V(t+1,p')]\}$, but $E[V(t+1,p')]$ will simply be $\frac{v}{1-\beta}-\int p f(p)dp$, which makes the $V(t,p)$ not a bellman equation(i.e. I eliminate the optimal choice from tomorrow in $V(t,p)$ equation.)
So how to write the Bellman Equation in such a situation? Thanks!
Maybe use $[t-T+1]^+$ as a multiplier for "losing": $$V(t,p)=\max\{\frac{v}{1-\beta},(-\frac{v\cdot M}{1-\beta}\cdot[t-T+1]^{+})+(u+\beta E_{\tilde{p}}[V(t+1,\tilde{p})]) \}$$ with some big $M>>2$.
This will give your original equations for all days $t<T$, but from day $T$ and on the agent has "lost" and suffer a definite penalty. (Note that $\displaystyle{\frac{v\cdot M}{1-\beta}>\frac{v}{1-\beta}+u+\beta E_{\tilde{p}}[V(t+1,\tilde{p})]}$)