Stochastic control with stopping times

Question

Given a wealth process that evolves as $$d w_t = r w_t dt + \theta_t ( \sigma dW_t + (\mu-r) dt) - c_t dt.$$ and smooth functions $u,F: [0, +\infty) \rightarrow \mathbb{R}$, how can we optimise the following:

$$V(w) = \sup_{c \geq 0, \, \theta, \, \tau} \mathbb{E} \bigg[ \int_0^{\tau} e^{- \rho t} u (c_t) dt + e^{-\rho \tau} F(w_{\tau}) \bigg| w_0 =w \bigg],$$ where $\tau$ is a stopping time.

The traditional method of using the HJB and martingale principle of optimal control does not seem to work in this case, when stopping time is involved.

Any suggestions on how to optimise this?

Answer 1

It seems this is a Infinite Time Horizon problem, it usually stops when the wealth reaches zero. Some information is missing here:

1. Wich parameter is the control. Is it $c_t$? it usually indicates the comsumption.
2. What $\mu$ and $r$ are. Are they functions?
3. What is $\theta_t$. Is this the control?
4. What is $W_t$, I will assume it is a standard Brownian motion. I will call it $B_t$ to make it less confusing.

Changing the notation $X_t$ for $w_t$ because in most books the variable $w$ is used for the candidates of the value function, and $X_t$ is used for stochastic processes, then the wealth ecuation is:

$$d X_t = (rX_t + \theta_t(\mu-r) - c_t) dt + \theta_t \sigma dB_t$$

Let's asume that $\theta$ and $c$ are the controls and they are independent. Also $c\geq 0$.

For infinite time horizon, the HJB equation in this case is (I'm changing the signs because the original HJB equation looks for the infimum): $$\beta w = \sup_{\theta \in \mathbb{R}, c\geq 0}((-rw + \theta(r-\mu) + c)w_x - \frac{1}{2}(\theta \sigma)^2w_{xx} - u(c))$$

For a constant $\beta > 0$. $w_x$ and $w_{xx}$ are the first and second derivatives of the candidate value function $w$. As long as $\theta$ and $c$ are independent I can do:

$$\beta w = -rww_x + \sup_{\theta \in \mathbb{R}}(\theta(r-\mu)w_x - \frac{1}{2}(\theta \sigma)^2w_{xx}) + \sup_{c \geq 0}(cw_x - u(c))$$

I asume the second derivative of $cw_x - u(c)$ with respect to $c$ is negative so, for the comsumption, the supremum is when: $$w_x = \frac{du}{dc_t}$$

Asuming $u$ is inyective:

$$c = (\frac{du}{dc_t})^{-1}(w_x)$$

For the control $\theta$: Again I asume the second derivative of $\theta(r-\mu)w_x - \frac{1}{2}(\theta \sigma)^2w_{xx}$ with respect to $\theta$ is negative, so its supremum is found at:

$$(r-u)w_x - \theta\sigma^2w_{xx} = 0$$ $$\theta = \frac{(r-u)w_x}{\sigma^2w_{xx}}$$

From here, to actually solve the problem, you propose an ansatz for $w$ and apply the verification theorem. It may happen that the problem has no known solution, or no solution at all.