I have difficulties writing down the following problem in a formal way:
We have some coke machines in a school and they can be working or being broken. So we have two states, working and broking.
(1) If it is broken, it costs us $8c$ and takes the whole day to repair it.
(2) If it is working we have two subcases: People using the machine or not. If nobody uses the machine we have no cost at all and the probability that the machine is not working the next day is $p$. If people use the machine we have cost $c$ and probability $q<(7/8)p$ that the machine is not working the following day.
Now we also have a discounted factor $\beta\in(0,1)$. We want to minimize the total-expected discounted cost over an infinite time.
$F(0)$ is the minimal cost if the machine is broken in the morning and $F(1)$ the minimal cost if machine is working. Now I want to show that it is optimal that nobody uses the machine onlfy if $\beta\le \frac{1}{7p-8q}$
I think the first thing to do is to write down the optimality equation, so $F(1)$ in terms of some integrals.
$F(1)=\int_0^{\infty}\min[x,\beta F(1)]p(x) dx=\beta F(1)+\int_0^{\infty}\min[x-\beta F(1),0]p(x)$
Is this the right way to start?
I assume that a broken machine must be repaired. If it is optimal to not use machine then expected lifetime discount cost on a day the machine is working is the discounted expected lifetime cost the following day:
$F(1)= \beta\{(1-p)F(1) + pF(0)\}$.
Expected lifetime discount cost on a day the machine is not working is the cost of repair that day plus the discounted expected lifetime cost the following day:
$F(0) = 8c + \beta F(1)$.
Solve to get
$F(1) = 8c\frac{\beta p}{(1-\beta)(1+\beta p)}$ and $F(0) = 8c\frac{\beta^2 p + (1-\beta)(1+\beta p)}{(1-\beta)(1+\beta p)}$.
Now the policy of not using the machine is optimal only if the expected discounted lifetime cost from doing so is no more than the expected discounted lifetime cost from using the machine. That is,
$\beta\{(1-p)F(1) + pF(0)\} \le c+\beta\{(1-q)F(1) + qF(0)\}$
or $\beta (p-q)\{F(0)-F(1)\} \le c$.
Substituting the expressions for $F(1)$ and $F(0)$ yields
$8c\frac{\beta(p-q)(1-\beta)}{(1-\beta)(1+\beta p)} \le c$
which results in the desired condition $\beta \le \frac{1}{7p-8q}$.