Markov chain problem in Ross's Introduction to probability models

782 Views Asked by At

It is example 4.10 and the problem states that a pensioners receives 2 at the beginning of the each month. The amount of money he needs to spend is independent of the amount he has and is equal to $i$ with probability $P_i, i=1,2,3,4, \sum_{i=1}^{4} P_i=1$. If the pensioner has more than 3 at the end of a month, he gives the amount greater than 3 to his son. The answer says the problem can a three states Markov chain. State 1 is at the end of the month the pensioner having less than or equal to 1 left, state 2 is having 2 left, and state 3 is having 3 left. The transition matrix is
$$ Q = \left| \begin{array}{ccc} 1 & 0 & 0 \\ P_3+P_4 & P_2 & P_1 \\ P_4 & P_3 & P_1+P_2 \end{array} \right| $$ My problem is at understanding $Q_{11}$. $Q_{11}$ says if the pensioner has less than or equal to 1 at the end the month, at the next end of the month the pensioner always end up with less than or equal to 1. But what I think is if the pensioner has 1 at the end of the month, at the next end of month the pensioner can have less than or equal to 1, or 2 by receiving 2 and spending 1. I am think the situation is more complicate than the above transition matrix and requires more states. Do I have any misunderstanding here? Thanks for the help.

1

There are 1 best solutions below

0
On BEST ANSWER

Quote: "we will let $1$ mean that the pensioner's end-of-month fortune has ever been less than or equal to $1$" (my emphasis). Hence $Q_{11}=1$.