I am presented with the following Markov chain example:
A trader sells large and expensive machines.
$X_n$ is the number of machines in stock at the start of week $n$.
$D_n$ is the number of machines demanded by customers during week $n$.
Assume that $D_n \sim \text{Poi}(3)$, the $D_n$ are independent, and that $D_n$ and $X_n$ are independent for each $n$.
There are two stipulations:
Inventory control: If $0$ or $1$ machines are left in stock by the end of a week, then machines are ordered and delivered to raise the stock to $5$ by the start of the next week. If $2$ or more machines are in stock at the end of the week, then no orders are placed.
Lost business: If $D_n > X_n$, then the unsatisfied demands are lost.
We seek to show that $X_n$ is a Markov chain.
$(X_n - D_n)^+$ is the number of machines in stock at the end of week $n$.
$$X_{n + 1} =\begin{cases} X_n - D_n & \text{if} X_n - D_n \ge 2 \\ 5 & \text{if} X_n - D_n \le 1 \end{cases}$$
$S = \{ 2, 3, 4, 5 \}$
Denote the history of the process up to time $n$ by $H_n = \{ X_0, X_1, \dots, X_n \}$.
Given that $X_n = i$, the independence assumptions ensure that $X_{n + 1}$ is conditionally independent of $H_{n - 1}$.
Case 1: $j = 2, 3,$ or $4$, and $i = j, \dots, 5$:
$$\begin{align} P(X_{n + 1} = j \vert X_n = i, H_{n - 1}) &= P(X_n - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n = j) \\ &= P(D_n = i - j) \\ &= e^{-3} \dfrac{3^{i - j}}{(i - j)!} \end{align}$$
Case 2: $i = 2, 3,$ or $4$, and $j = 5$:
$$\begin{align} P(X_{n + 1} = 5 \vert X_n = i, H_{n - 1}) &= P(X_n - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1 \vert X_n = i, H_{n - 1}) \\ &= P(i - D_n \le 1) \\ &= P(D_n \ge i - 1) \\ &= 1 - P(D_n \le i - 2) \\ &=: p_{i5} \end{align}$$
Case 3: $i = 5, j = 5$:
$$p_{55} = P(D_n = 0) + P(D_n \ge 4) = P(D_n = 0) + 1 - P(D_n \le 3).$$
Building on this example, I am also presented with this example:
Assume he pays \$1500 per machine, and sells them for \$1750, and that he incurs a cost of \$50/week for storage. If he has five machines in stock now, what is his expected profit over the next 10 weeks?
Answer. Recall that $S = \{ 2, 3, 4, 5 \}$. If $X_n = j$, then there are $j$ machines in stock at the start of week $n$, and the profit for that week is
$$C(j) = -50j + (1750 - 1500)\min(j, D_n),$$
since he cannot sell more than $j$ machines. hence
$$c(j) = -50j + 250E[\min(j, D_n)].$$
The expectation here equals $\sum_{i = 0}^j iP(D_n = i) + jP(D_n > j),$ and the probabilities required for this calculation can be found in a table of Poisson probabilities or a suitable software package.
How did the authors get that the expectation equals $\sum_{i = 0}^j iP(D_n = i) + jP(D_n > j)$?
I would greatly appreciate it if people would please take the time to clarify this.
By “the expectation”, they’re referring to the expression $E[\min(j,D_n)]$. This minimum is $i$ if $D_n=i$ with $0\le i\le j$, and $j$ if $D_n\gt j$. The expectation is the sum of these values weighted by the probabilities of their occurrence.