Markov Chain, finding the steady state vector

794 Views Asked by At

Suppose that if it is sunny today, there is a 60% chance that it will be sunny tomorrow, a 30% chance that it will be partly cloudy and a 10% chance that it will be completely cloudy. If it is partly cloudy today, there is a 40% chance that it will be sunny tomorrow and a 30% chance that it will be partly cloudy. If it is completely cloudy today, there is a 40% chance that it will be sunny tomorrow and a 50% chance that it will be partly cloudy.

Determine in the long term, what is the probability that it will be sunny in the next day.

I used the information above to create a stochastic matrix that is:

$$ A = \begin{bmatrix} 0.6 & 0.4&0.4\\0.3&0.3&0.5\\0.1&0.3&0.1\end{bmatrix}$$

How would I determine the steady-state vector from this matrix?

1

There are 1 best solutions below

0
On

So far you have constructed a table, not a matrix. Your table does not become a matrix until there is a significance to matrix multiplication.

Let $S_n$ be the probability that it is sunny on day $n$.

Let $P_n$ be the probability that it is partly cloudy on day $n$

Let $C_n$ be the probability that it is completely cloudy on day $n$.

Suppose that you know $S_k$, $P_k$, and $C_k$. Find a formula, for example, for the probability that it is sunny the next day:

$$S_{k+1} = A_{s \to s}S_k + A_{p \to s}P_k + A_{c \to s}C_k \tag{A}$$

where $A_{x \to y}$ is the probability that the weather transitions from $x$ to $y$. Formula (A) should look like something to you from forlorn memories of linear algebra lectures.

This should allow to write the weather of one day in terms of the weather of a previous day using linear algebra. Then extrapolate to infinity.