How often is the room cleaned on average? (Stochastic/Probability question)

Question

If the last time Bob cleaned up his room was $k$ days ago, then he cleans it up today with probability $1/(k + 1)$. But if the last time he cleaned up was 4 days ago, his father cleans it up. How often is the room cleaned on average?

I'm not really sure what to do for this question and have been stuck for a while. Would anyone be able to help out?

Answer 1

The first step is plainly to figure out what the states are. We have $C_k$ for $k=1,2,3$ meaning that the room was last cleaned $k$ days ago. We also have two absorbing states, $B$ and $F$, meaning that Bob or his father cleans the room, respectively. We know that the initial state is $C_1$.

If the chain is in state $C_k,\ k=1,2$ then it moves to state $B$ with probability $1/(k+1)$ and to state $C_{i+1}$ with probability $k/(k+1)$. If it is in state $C_3$ then it moves to state $F$ with probability $3/4$.

Now you can find the probability of states $B$ and $F$ by analyzing the absorbing Markov chain. But this is overkill. No state is ever re-visited, so you can just draw a tree diagram.

Answer 2

1st interpretation
There's basically two ways to solve this. I only show the longer way because the problem breaks / involves a null recurrent chain.

Longer way
$P = \pmatrix{ q_1 & p_1 & 0 & 0 & 0 & 0 & \dots\\ q_2 & 0 & p_2& 0 & 0 & 0 & \dots\\ q_3 & 0 & 0 & p_3 & 0 & 0 & \dots\\ q_4 & 0 & 0 & 0 & p_4 & 0 & \dots\\ q_5 & 0 & 0 & 0 & 0 & p_5 & \dots\\ q_6 & 0 & 0 & 0 & 0 & 0 & \ddots\\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \ddots}$

where $q_k = \frac{1}{1+k}$
while longer, this is standard /canonical matrix for renewal processes on the natural numbers known as an Age matrix, so this is worth learning. The steady state probability of being in state one is given by $\frac{1}{\mu}$ where $\mu$ is the expected time of an epoch (expected time until visiting state 1 again given a start in state 1)
$\mu = 1 + \sum_{k=1}^\infty\big(\prod_{j=1}^k p_j\big) = \sum_{k=0}^\infty Pr\{X\gt k\} $

But this telescopes

$\mu = 1 + \sum_{k=1}^\infty\big(\prod_{j=1}^k p_j\big) = 1 + \sum_{k=1}^\infty\big(\prod_{j=1}^k \frac{k}{k+1}\big)= 1 + \sum_{k=1}^\infty\frac{1}{k+1} = \infty $

i.e. we recover the harmonic series. This means $P^k \to \mathbf 0$. (We can actually calculate and show the chain is null recurrent not transient, but this isn't needed here).

So the answer would be that the room is cleaned every time state 1 or 4 is visited but the time averaged amount that occurs is zero because this isn't a positive recurrent chain.