I had a little bit about stochastic processes during my "Statistical Physics" course and on my exam I got a problem with a Markov chain. My solution seems to be without computational mistakes (checked it with Mathematica), so I suspect some deeper misconception, because the components of my probability vector don't add up to 1 after my calculations. Below I present the problem and my solution. I'll be grateful for any suggestions.
A 2 state particle is initially in a state $P(0)=(1,0)^T$. It's transition matrix is:
$$ T = \left( \begin{array}{cc} 5/6 & 1/6 \\ 2/6 & 4/6 \\ \end{array} \right) $$
Calculate the probability of the particle being in its second state after 5 steps.
My solutions in the following:
- find the eigenvalues $(1$, $\frac{1}{2})$ and their respective eigenvectors $((1,1)^T$, $(1,-2)^T)$
- find the $P'(0)$ (initial probability vector's representation in $T$'s eigenvector basis):
$$ P'(0)=\beta^{-1} \cdot P(0) = \left( \begin{array}{cc} 2/3 & 1/3 \\ 1/3 & -1/3 \\ \end{array} \right) \cdot \left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right) = \left( \begin{array}{c} 2/3 \\ 1/3 \\ \end{array} \right) $$
- calculate $P'(5) = T'^5\cdot P'(0)$:
$$ \left( \begin{array}{cc} 1 & 0 \\ 0 & (1/2)^5 \\ \end{array} \right) \cdot \left( \begin{array}{c} 2/3 \\ 1/3 \\ \end{array} \right) = \left( \begin{array}{c} 2/3 \\ 1/96 \\ \end{array} \right) $$
- return to the original basis with $P(5) = \beta \cdot P'(5)$:
$$ \left( \begin{array}{cc} 1 & 1 \\ 1 & -2 \\ \end{array} \right) \cdot \left( \begin{array}{c} 2/3 \\ 1/96 \\ \end{array} \right) = \left( \begin{array}{c} 65/96 \\ 62/96 \\ \end{array} \right) $$
Now, as I understand the vector $P(5)$ gives us 2 probabilities: one, of the particle being in state A after 5 steps ($65/96$), and second, of the particle being in state B after 5 steps ($62/96$). But those are the only two options, so those numbers should add up to 1. So what did I do wrong?
$T = \begin{pmatrix}5/6 & 1/6\\ 2/6 & 4/6\end{pmatrix}$ is a right stochastic matrix (see the Wikipedia entry, for example). So you need to right-multiply the state vector (which is a row vector rather than a column vector as you have given) with $T$. This gives, for example:
$(1\quad 0)\begin{pmatrix}5/6 & 1/6\\ 2/6 & 4/6\end{pmatrix} = (5/6\quad 1/6)$ (whereas $T(1 \quad 0)^T$ gives $(5/6 \quad 1/3)$, which is not a state vector).
Similarly, $(1 \quad 0)T^5 = (65/96 \quad 31/96)$.