It is example 4.10 and the problem states that a pensioners receives 2 at the beginning of the each month. The amount of money he needs to spend is independent of the amount he has and is equal to $i$ with probability $P_i, i=1,2,3,4, \sum_{i=1}^{4} P_i=1$. If the pensioner has more than 3 at the end of a month, he gives the amount greater than 3 to his son. The answer says the problem can a three states Markov chain. State 1 is at the end of the month the pensioner having less than or equal to 1 left, state 2 is having 2 left, and state 3 is having 3 left. The transition matrix is
$$
Q = \left| \begin{array}{ccc}
1 & 0 & 0 \\
P_3+P_4 & P_2 & P_1 \\
P_4 & P_3 & P_1+P_2 \end{array} \right|
$$
My problem is at understanding $Q_{11}$. $Q_{11}$ says if the pensioner has less than or equal to 1 at the end the month, at the next end of the month the pensioner always end up with less than or equal to 1. But what I think is if the pensioner has 1 at the end of the month, at the next end of month the pensioner can have less than or equal to 1, or 2 by receiving 2 and spending 1. I am think the situation is more complicate than the above transition matrix and requires more states. Do I have any misunderstanding here? Thanks for the help.
2026-04-24 08:09:01.1777018141
Markov chain problem in Ross's Introduction to probability models
782 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
Quote: "we will let $1$ mean that the pensioner's end-of-month fortune has ever been less than or equal to $1$" (my emphasis). Hence $Q_{11}=1$.