Markov probability matrix and steady state solutions

83 Views Asked by At

I am currently doing a question and can seem to find where my problem lies with my final solution. The question is based on markov probability and steady state solutions, but the solution I get for the steady state the probability don't add to one, and I cant see where I have gone wrong wheather is a miss calculation or my understanding or Markov matrices is incorrect.

The question is quite long as one part leads on to the other I will display the question and my final solution for each section to save some white space.

Part 1 enter image description here

For this I used the standard equation of the form $(M-b)\bar{x}=0$ to find the eigenvalues, then subbed the eigenvalue back into the equation $(M-b)\bar{x}=0$ to find the eigenvectors.

To which my final results were.

$$\lambda _1=a+b,\:\:\bar{x}=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\ 1\end{pmatrix}$$

$$\lambda _2=a-b,\:\:\bar{x}=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\ -1\end{pmatrix}$$

part 2

enter image description here

For this I used the conditional probability to form the matrix

$$\begin{pmatrix}p&1-p\\ 1-p&p\end{pmatrix}$$

part 3 enter image description here

I first used the general equations for $\lambda_1$ and $\lambda_2$ to get the eigenvalues

$$\lambda_1=1$$ $$\lambda_2=2p-1$$

Using the same for the eigenvectors obtained from part 1. I then went on to from $T$

$$T=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&2p-1\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}$$

Part 4 enter image description here

using the above equation I formed the steady state solution

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1^n&0\\ 0&\left(2p-1\right)^n\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}u_0\\ v_0\end{pmatrix}$$

as $n \rightarrow \infty$ then I got

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&0\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}u_0\\ v_0\end{pmatrix}$$

by subbing in the condition for i)

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&0\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}0.2\\ 0.8\end{pmatrix}$$

the state vector comes out to be $\begin{pmatrix}0.25\\ 0.25\end{pmatrix}$ which cannont be correct at the state vector should sum to 1.

Could someone maybe expand upon where I have gone wrong. I understand this is a long post but even if someone could point to the part of the question where my mistake is and just give a little explanation.