Markov chain - is my diagram/matrix correct?

61 Views Asked by At

A boy goes to school on a bike or on foot.

If one day he goes on foot, then on the second day he takes a bike with probability $0.8$.

If he goes on a bike one day, then he falls off the bike with probability $0.3$ and goes on foot the next day.

What is the probability that one the last day of school he goes to school on a bike?

We assume that the school year has just started.

I can see that we need to define a Markov chain.

I've tried coming up with a directed graph which I cannot draw here, but I've pasted it here

This is what the matrix would look like:

$$A= \begin{array}{l} \mbox{bike} \\ \mbox{falls} \\ \mbox{on foot} \\ \mbox{doesn't fall} \end{array} \left( \begin{array}{cccc} 0 & 0.3 & 0 & 0.7 \\ 0 & 0 & 1 & 0\\ 0.8 & 0 & 0.2 & 0 \\ 1 & 0 & 0 & 0 \end{array} \right) $$

This matrix is ergodic and solving the equation $[x_1, x_2, x_3, x_4]A = [x_1, x_2, x_3, x_4], \ \ x_1 + x_2 + x_3 +x_4=1$ I get that $x_1=8/19$.

So it would seem that the probability that on the last day of school the boy travels on a bike is $8/19$.

Is that true? Is my solution correct?

1

There are 1 best solutions below

5
On BEST ANSWER

I think you can simplify things a lot by just using two states, foot (state $1$) and bicycle (state $2$). Then you have the transition matrix: $$\begin{pmatrix} 0.2 & 0.8 \\ 0.3 & 0.7 \end{pmatrix}$$

In fact, the way you've done it is a bit confusing because falling off a bicycle is not a state in the same sense that going to school by foot/bicycle is. For one time step (a day) his state is the mode of transport he uses. But he doesn't spend a full day falling off a bike, so it doesn't make sense to use this as a state.