During a 3h shift, an employee estimates that the probability for him to drink a cup of coffee rises at a constant rate $\omega=0.8/hour$. Given that the rate is time-independent, he expects an average of 2.4 cups consumed in the whole shift. No cup has been consumed at the beginning of the shift ($P_n(t=0)=\delta_{n 0}$).
I want to model this problem by considering Markovian time evolution for the probabilities in order to keep track of fluctuactions.
This is my attemp:
Probability vector whose components $p_i$ give the probability that $i$ have been consumed after the 3h shift at a given time $t>0$.
$$ P(t)=\begin{bmatrix} p_{0}(t) \\ p_{1}(t)\\ p_{2}(t)\\ p_{3}(t)\\ \end{bmatrix} $$
The transition matrix, in left stochastic notation, with elements given by:
$$(W)_{ij}=w_{ij}-\delta_{ij}\sum_{k=0}^{3}w_{kj}, w_{ii}=0$$
where $w_{ij}$ are the non-negative rates for transition from state $j$ to state $i$ per unit time. In this case, the transition rate from having $j$ cups of coffe to $i$ cups.
In this problem I have the following transition rates: $$w_{ij}=0 \hspace{1cm} j>i \\ w_{i+1,i}=0.8\\ w_{i+2,i}=0.8/2=0.4\\ w_{i+3,i}=0.8/3=0.27$$\
I'm considering that there is not possibility to "untake" a cup a coffe, $w_{ij}=0$ for $j>i$ and that if we have 1 cup of coffe, the probability to take 2 more cups of coffe in the next hour ($w_{i+2,i}\rightarrow w_{31}$) will be half the probability to take 1 cup. I don't know if this assumption makes sense, but I couldn't think of anything better with the information "the probability for him to drink a cup of coffee rises at a constant rate $\omega=0.8/hour$" I have been given.
I end up with this transition matrix: $$ M= \left( {\begin{array}{cc} -1.47 & 0 & 0 & 0 \\ 0.8 & -1.2 & 0 & 0 \\ 0.4 & 0.8 & -0.8 & 0 \\ 0.27 & 0.4 & 0.8 & 0 \\ \end{array} } \right) $$
Now the next step would be to solve the master equation: $$\frac{d}{dt}P(t)=W·P(t)$$
Nevertheless, I'm not very confident with the transition matrix I got because I am not sure the transitions rates I wrote make sense. Any comments on this? Would you model it in a different way?
Your construction could be valid in a discrete case, although you say that the rate is constant so it shouldn't depend on which states it's jumping between. This is a classic example of a birth-death process with constant "birth" rate $\omega=0.8$ and "death" rate equal to zero since the number of cups of coffee always increases. Here you can picture the variation in the number of cups of coffee with an influx from $n-1\to n$ and a probability outflux from $n\to n-1$. You can solve it using a continuous time master equation for the probability of drinking $n$ cups of coffee at any given time $t$: $$\frac{d}{d t}p_n(t) = \omega[p_{n-1}(t)-p_n(t)],$$ which is solved by $$ p_N(n,t) =\frac{(0.8t)^n}{n!}e^{-0.8t}.$$