Markov Process: predict the weather using a stochastic matrix

Question

I have the following stochastic matrix

$$ P = \begin{pmatrix} P(S \mid S) = 0.5 & P(F \mid S) = 0.2 & P(R \mid S) = 0.3 \\ P(S \mid F) = 0.2 & P(F \mid F) = 0.7 & P(R \mid F) = 0.1 \\ P(S \mid R) = 0.75 & P(F \mid R) = 0.15 & P(R \mid R) = 0.1 \end{pmatrix} $$

Where $S$ means sunny, $R$ means rainy and $F$ means foggy. For example $P(S \mid F)$ is the probability of being sunny tomorrow given that today is foggy. Similarly, $P(R \mid R)$ is the probability of tomorrow being rainy if today is also rainy.

The graph representing the situation would look like this

Now suppose today is Monday and it's sunny. I would like to estimate the probabilities of

1. a sunny Tuesday

2. a sunny Tuesday and Wednesday

3. a sunny Wednesday

On Wikipedia there's a similar problem, but since I am new to these things, I decided to ask you help you anyway at least for some clarifications.

As far as I have understood a Markov process is a memoryless stochastic process, that only remembers the current state in order to predict future states.

For point $1$, we can simply use the fact that $P(S \mid S) = 0.5$, i.e. the probability of tomorrow being sunny if today is sunny is $0.5$, which is therefore the answer.

For the other cases it's a little bit more complicated: we cannot use directly any value from the stochastic matrix $P$. We need of course to do intermediary calculations.

The example I am linking to, to predict the weather on day $1$, it creates a vector representing the weather on day $0$. Similarly, in my case, I think that vector would be $$v_1 = \begin{pmatrix} 1 & 0 & 0 \end{pmatrix}$$ that is the $1$ represents that it's $100 \%$ sunny. If I multiply $v_1$ by $P$ I obtain (similarly to the Wikipedia's problem above) the following vector $$v_1 =\begin{pmatrix} 0.5 & 0.2 & 0.3 \end{pmatrix}$$

From this to obtain my guess of $0.5$ there's some distance of reasoning. Could you please explain it to me

1. How do you explain the $0.5$ I am guessing (if it's correct) from the calculations that I just did and done in the solutions to the Wikipedia's problem. If it's not correct, why and how can I solve it?

2. In the third question, do I need to do the same thing as I did for the first point, but using $P^2$, like in the Wikipedia's article?

3. The second question involves a compound probability (I think). Am I right? How exactly would I proceed to solve this case, and why?

Thanks for any help (and sorry for the long post)!

Answer 1

The way you have written out the matrix $P$, we pre-multiply them with a row vector, and obtain another row vector that represents the next day's weather distribution. In general, we say

$$ v_{i+1} = v_i P $$

where $v_i$ represents the $i$th day's weather distribution. In this case, we have $v_1 = \left[ \begin{array}{ccc} 1 & 0 & 0 \end{array} \right]$, and then

\begin{align} v_2 & = v_1 P \\ & = \left[ \begin{array}{ccc} 1 & 0 & 0 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] \\ & = \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \end{array} \right] \end{align}

which tells us that Tuesday will be sunny with probability $0.5$.

We could, as you suggest, do the same thing with $P^2$ to obtain Wednesday's weather distribution, but it might be easier simply to apply $P$ to $v_2$:

\begin{align} v_3 & = v_2 P \\ & = \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] \\ & = \left[ \begin{array}{ccc} 0.515 & 0.285 & 0.2 \end{array} \right] \end{align}

which tells us that Wednesday will be sunny with probability $0.515$.

It is simplest to answer the middle question—the probability that it will be sunny both Tuesday and Wednesday—by simply observing that it is sunny on Tuesday with probability $0.5$ (as we already observed), and then identical reasoning tells us that conditioned on that fact, it will also be sunny on Wednesday with probability $0.5$, and multiply those together to obtain $0.25$.

Alternatively, one could eliminate the rainy and foggy probabilities from $v_2$, and inspect the first component of

$$ \left[ \begin{array}{ccc} 0.5 & 0 & 0 \end{array} \right] \left[ \begin{array}{ccc} 0.5 & 0.2 & 0.3 \\ 0.2 & 0.7 & 0.1 \\ 0.75 & 0.15 & 0.1 \end{array} \right] = \left[ \begin{array}{ccc} 0.25 & 0.1 & 0.15 \end{array} \right] $$

to obtain the same answer. Note that this "clipped" vector is not a probability distribution (it does not sum to $1$), but that does not matter for our present purposes.