This is one of my questions in Stochastic Process: A store that stocks a certain commodity uses the following $(s, S)$ ordering policy; if its supply at the beginning of a time period is $x$, then it orders $$ \left\{\begin{array}{l} 0 \quad x \geq s \\ S-x \quad x<s \end{array}\right. $$ The order is immediately filled. The daily demands are independent and equal $j$ with probability $\alpha_j$. All demands that cannot be immediately met are lost. Let $X_n$ denote the inventory level at the end of the nth time period. Argue that $\left\{X_n, n \geq 1\right\}$ is a Markov chain and compute its transition probabilities. This is a very simple question i thought. And i think the answer is(which is from http://www.charmpeach.com/category/stochastic-processes/page/2/):
Let $Y_i$ denote the demand of the ith day, then $$ X_n= \begin{cases}X_{n-1}-Y_n & X_{n-1} \geq s \\ S-Y_n & X_{n-1}<s\end{cases} $$ Since $Y_i$ is independent, $\left\{X_n, n \geq 1\right\}$ is a Markov chain, and its transition probabilities are $$ P_{i j}=\left\{\begin{array}{l} \alpha_{i-j} \quad i \geq s, i \geq j \\ \alpha_{S-j} \quad i<s \\ \end{array}\right. $$
However, i see another solution considers $j=0$ situation which makes me very confused:
Denote by $O_n$ the amount of order and $D_n$ the amount of demand at the beginning of period $n$. We have $$ X_{n+1}=\left(X_n+O_n-D_n\right)^{+} $$ where $$ X_n+O_n= \begin{cases}S & X_n<s \\ X_n & X_n \geq s\end{cases} $$ and $D_n$ is independent to the history $\left(X_n, X_{n-1}, \ldots\right)$. So $\left(X_n\right)$ is a Markov chain. $$ P_{i, j}=\operatorname{Pr}\left(X_{n+1}=j \mid X_n=i\right)=\operatorname{Pr}\left(\left(i+O_n-D_n\right)^{+}=j\right) $$ If $j=0$, we have $$ P_{i, j}=\operatorname{Pr}\left(D_n \geq i+O_n\right)= \begin{cases}\operatorname{Pr}\left(D_n \geq S\right)=\sum_{k=S}^{\infty} \alpha_k & i<s \\ \operatorname{Pr}\left(D_n \geq i\right)=\sum_{k=i}^{\infty} \alpha_k & i \geq s\end{cases} $$ If $j>0$, we have $$ P_{i, j}=\operatorname{Pr}\left(D_n=i+O_n-j\right)= \begin{cases}\operatorname{Pr}\left(D_n=S-j\right)=\alpha_{S-j} & i<s \\ \operatorname{Pr}\left(D_n=i-j\right)=\alpha_{i-j} & i \geq s\end{cases} $$
I don't make sense why we need to consider $j=0$ and why the answer is not same as above. Which solution is correct? I would be very appreciate if you can give me any comments.
Your solution does not take into account the sentence
which corresponds to the intuitive constraint that the stock cannot become negative. So the correct expression for $X_n$ is: $$X_n = \begin{cases}X_{n - 1} - Y_n & \text{if } X_{n-1} \ge s \text{ and } X_{n - 1} > Y_n\\ S - Y_n & \text{if } X_{n - 1} < s \text{ and } S > Y_n \\ 0 & \text{otherwise.}\end{cases}$$ You should be able to deduce that $X_n$ is a Markov process and the correct transition probabilities from here.