Professor has 4 umbrellas, Markov chain and Probability

Question

OK this problem is making me tear my hair out. I need someone to walk me through this in baby-steps method like 1 + 1 = 2. I am trying to figure out what I don't understand. I know this is going to be a Markov chain.

Answer 1

Step one is to write down the four by four transition matrix: the probability of moving from one state to another. Step two is to find the limit as that matrix works on itself.

For example, with one umbrella at home, there is a $p$ chance of rain, but even if it does, and he takes the umbrella to work, he will take it back. He can't take more than one umbrella, so he can't transition to states 3 or 4. So the only possible transitions are to states 1 or 2. With probability $1-p$ it doesn't rain in the morning, so with probability $(1-p)p$ it doesn't rain in the morning but rains that night, meaning he leaves one umbrella at home but brings one back from work, giving him two umbrellas at home. So the vector of transitions from state 1 into states 1,2,3,4 is $(1-p(1-p),p(1-p),0,0)$. Similar Logic should work for the other transitions, taking the state as the number of umbrellas at home in the morning.

Answer 2

As was noted above you have to construct the transition matrix.

Say there is one umbrella in the morning at his house. How can he end up with one umbrella at his house the next morning? Lets look at the 4 cases:

Rains morning and night: brings umbrella to office and then back hence remains with 1 at home

Rains morning but not night: brings umbrella to office but also back because otherwise office has all 4, hence remains with 1 at home

Does not rain morning, rains night: does not bring umbrella in the morning does bring one back at night so ends up with 2 at home

Does not rain either morning or night: does not bring umbrella either time hence remains with 1 at home

Hence the transition probability $ 1 \to 1$ is $M_{11}=1-p(1-p)$. What is the transition probability $1 \to 2$? That only happens in the one case outlined above hence $1 \to 2$ has probability $M_{12} = p(1-p)$. The transitions $1 \to 3$ and $1 \to 4$ have zero probability.

So that's your first row of the transition matrix as you requested. Now fill in the rest.

Answer 3

Assume you are at home and going to office. There can be 5 states either you have 01234 Umbrellas at home. There are four events possible : (rain rain) ; (Rain, no rain) ; (no rain, rain) ; (no rain,no rain).

you can construct a probability transition matrix in the case of each of these events starting from every state. Then you can calculate the steady state probabilities of All the five states.Then,

P(wet) = s0 * P(rain),

Since the case going from office to home is symmetrical and the probability of being at office or being at home initially is equal the final answer is the above expression only.

upon calculation I found that (s1 = s2 = s3 = s4 = 1/(q + 4)) and s0 = (q/(q + 4)) . Where q = 1- p.