Singular Control Construction for HJB Equation on Poisson Process

71 Views Asked by At

The setup: We have a Poisson Random Process, with unit increments (rate $\mu_1>0$) and decrements (rate $\mu_2>0$), whose state is denoted by $x\in\mathbb{N}_0$. We are trying to solve an infinite horizon, continuous time, minimal discounted cost problem (rate $\rho>0$), with a singular control $u$ that, at the instant of a jump, allows us to instantaneously reduce the state $x$ by $u\in\{0,1,\dots,x\}\equiv \mathcal{U}^x$, paying some cost $bu$. There's also a running cost of $ax$. We are allowed to control the system only at the times of the jump. We have $b>0$, $a>0$.

Let $V$ be the value function. After some work, I showed that the HJB equation for the problem is:

$$\small ax+\inf_{u\in\mathcal{U}_x}\{(b-a)u+\mu_1(V(x-u+1)-V(x))+\mu_2\mathbb{I}_{x-u\geq 1}(V(x-u-1)-V(x))\}-\rho V(x)=0$$

The task at hand now (and I'm really stuck) is to identify conditions on the problem data such that there exists $\overline{x}\in\mathbb{N}_0$, such that the optimal control $u^*$ has the following structure: $$u^*(x)=\begin{cases}0 &\text{if } x\leq\overline{x} \\x-\overline{x}&\text{if } x>\overline{x}\end{cases}$$

My thoughts: First, the infimum is always achieved because $\mathcal{U}^x$ is finite. Inside the infimum, we have a linear term and a functional term. I think I'm looking for conditions such that the linear term and functional terms pull in opposite directions as $x$ increases. I want, for low values of $x$, the $(b-a)u$ term to dominate, so by setting $(b-a)<0$, $u^*=0$ is optimal. I want, for the opposite case, the functional part to dominate, so that the optimum is achieved at $u^*>0$. Now, the question is how to formalize this. I feel like I need to make an argument of monotonicity of $V$, especially to characterize a threshold $\overline{x}$. Any thoughts are more than welcomed. Thanks!