Formulate the following situation as a CTMC and solve the balance equations for the steady-state probabilities.
Suppose demand for an item in inventory at a warehouse is a Poisson process at rate $\lambda$, i.e. arriving customers are Poisson where each customer demands one unit. When the inventory hits zero, an order of constant size $S$ units is placed with the factory. Delivery time of the entire order is exponential with rate $\mu$. During this time, demand is lost at a cost of $\$c/\text{unit}$. The inventory holding cost is $\$h/\text{unit}/\text{unit time}$, based on the average amount of inventory held.
There is much going on here. I am not sure how to determmine the CTMC states. I guess make the states the demand? I don't know if the prices and other thing matter. I think the prices are just extra information, don't need them to make the Markov chain. Am I right? Not really sure where $\mu$ will go either.
For example I guess state 0 to state 1 has transition rate $\lambda$. State $n$ to state $n + 1$ has transition rate $\lambda$ too. Is this ok? Where I can put transitions with $\mu$?
Maybe I should make the states the inventory left?
Thanks
Let $X(t)$ be the number of items in the warehouse at time $t$. Then $\{X(t):t\geqslant 0\}$ is a continuous-time Markov chain on state space $\{0,1,\ldots,S\}$ with transition rates $$ q_{ij} = \begin{cases} \lambda,& j=i-1, j>0\\ \mu,& (i,j) = (0,S) \end{cases} $$ We have the balance equations for the stationary distribution $\lambda\pi_i=\mu\pi_0$ for $1\leqslant i\leqslant S$, hence $\pi_i = \frac\mu\lambda\pi_0$ and from $\sum_{i=0}^S\pi_i=1$ we have $$ \pi_0+S\frac\mu\lambda=1 \implies \pi_0 = \frac\lambda{\lambda+S\mu},\quad \pi_i = \frac\mu{\lambda+S\mu},\ 1\leqslant i\leqslant S. $$