What is the limiting distribution of this continuous time Markov chain?

352 Views Asked by At

Consider the following inventory model. You are managing an inventory of bikes. Demand for bikes arrives as a rate 1 Poisson process. Whenever a demand arrives, it is always a demand for exactly one bike. At any given time t, you have two types of inventory - inventory on hand $H_t$, and inventory ordered but not yet arrived $O_t$. When a demand arrives, if you have at least one bike on hand (i.e. $H_t \ge 1$), you meet the demand with the inventory on hand, i.e. you give the bike to the customer and $H_t$ is reduced by 1. If you have no inventory on hand and a demand comes, AND you have either 0 or 1 bikes in inventory ordered but not yet arrived (i.e. $O_t \le 1$), you lose that customer’s demand, and place an order for a new bike. A newly ordered bike takes an i.i.d. Expo(2) amount of time to arrive. If you have no inventory on hand and a demand comes, and you have at least 2 bikes in inventory ordered but not yet arrived, you lose that customer’s demand, and do not place any new orders. Suppose that a demand arrives at some time $t\gg 0$. What is the probability that this customer receives a bike?

I tried to model this problem by CTMC with states of double-index. That is, state $(i,j)$ for $i\ge 0$ and $0\le j\le 2,$ where $i$ indicates for number of $H_t$ and $j$ indicates for number of $O_t$. And let $A$ be generating matrix of this Markov chain and find $\eta A=0$ with all entries of $\eta$ are positive and $\sum_{(i,j)}\eta_{(i,j)}=1$. Then the answer should be $1-\sum_{j=0}^2\eta_{(0,j)}$. But the system of $\eta A=0$ is too complicated to find a solution. I wonder if there is some other way to look at it.