I'm working through a problem from Probabilistic Robotics by Thrun, Burgard and Fox. Chapter 2 Q2 is as follows;
I'm trying to solve part (d). For part (c) I wrote a numeric python script and ended up with;
$$ p(Sunny) \approx 0.642 \\ p(Cloudy) \approx 0.287 \\ p(Rainy) \approx 0.071 $$
For part (d), I used the theorem of total probability to derive a series of equations for the state transitions;
$$ \begin{align} \begin{bmatrix} p(Sunny) \\ p(Cloudy) \\ p(Rainy) \end{bmatrix} &= \begin{bmatrix} p(Sunny) \\ p(Cloudy) \\ p(Rainy) \\ \end{bmatrix} \begin{bmatrix} 0.8 & 0.4 & 0.2 \\ 0.2 & 0.4 & 0.6 \\ 0 & 0.2 & 0.2 \\ \end{bmatrix} \\ \vec{p} &= \vec{p} \begin{bmatrix} 0.8 & 0.4 & 0.2 \\ 0.2 & 0.4 & 0.6 \\ 0 & 0.2 & 0.2 \\ \end{bmatrix} \\ \vec{0} &= \begin{bmatrix} -0.2 & 0.4 & 0.2 \\ 0.2 & -0.6 & 0.6 \\ 0 & 0.2 & -0.8 \\ \end{bmatrix} = A \end{align} $$
I then tried to invert this matrix, which it turns out does not have an inverse (I get a determinant of 0). I read in an online problem set that an alternative way to solve for the stationary distribution is to square the matrix A until you arrive at a stable matrix (all rows are identical). Doing this in Matlab leads to the matrix
$$ \begin{bmatrix} Inf & -Inf & Inf \\ -Inf & Inf & -Inf \\ Inf & -Inf & Inf \\ \end{bmatrix} $$
- Have I made a mathematical error somewhere in trying to solve this problem?
- Is it possible that the point of this question is that there is no analytical solution to the stationary distribution? How would I recognize this scenario more directly in the future?