Calculating probability in a Markov Chain

Question

Suppose I have this Markov chain:

And suppose that:

$P_{AA} = 0.70$

$P_{AB} = 0.30$

$P_{BA} = 0.50$

$P_{BB} = 0.50$

I realize that $P_{AA} + P_{AB} = P_{BA} + P_{BB}$ but when I simulate I'm getting some percentage of A's and B's. I'm trying to calculate the probability (i.e. the predicted percentage of A's and B's) to check this data, so I want to calculate $P(A)$ and $P(B)$.

My stochastics is very rusty, so I'm sure someone with a basic understanding of probability and stochastics would know how to calculate this.

EDIT:

$ \pi_A = 0.7 * \pi_A + 0.5 * \pi_B $

$ \pi_A - 0.7 * \pi_A = 0.5 * \pi_B $

$ \pi_A = \frac{0.5}{0.3} * \pi_B $

$ \frac{0.5}{0.3} \pi_B + \pi_B = 1 \hspace{10 mm} since \hspace{5 mm} \pi_A + \pi_B = 1 $

$ \pi_B = 0.375 = \frac{3}{8} $

$ \pi_A = 0.625 = 0.7*0.625 + 0.5*0.375 $

$ \pi_B = 0.375 = 0.3*0.625 + 0.5*0.375 $

These helped a lot in solving this problem:

http://www.haverford.edu/econ/econ365/Note%20on%20Markov%20Chains.pdf

http://www.mast.queensu.ca/~stat455/lecturenotes/set3.pdf

Answer 1

I'm learning Markov Chains from the ground up, so I needed a step-by-step solution. The formulas and theory were easy to find online but that's not why I posted and I specified that in the original post. So here is how I went about solving it, and it would be great if this helps others who might have gotten a little lost or confused at the early stages of the problem (and if not then you're doing better than me).

$ \pi_A = 0.7 * \pi_A + 0.5 * \pi_B $

$ \pi_A - 0.7 * \pi_A = 0.5 * \pi_B $

$ \pi_A = \frac{0.5}{0.3} * \pi_B $

$ \frac{0.5}{0.3} \pi_B + \pi_B = 1 \hspace{10 mm} since \hspace{5 mm} \pi_A + \pi_B = 1 $

$ \pi_B = 0.375 = \frac{3}{8} $

$ \pi_A = 0.625 = 0.7*0.625 + 0.5*0.375 $

$ \pi_B = 0.375 = 0.3*0.625 + 0.5*0.375 $

These helped a lot in solving this problem:

http://www.haverford.edu/econ/econ365/Note%20on%20Markov%20Chains.pdf

http://www.mast.queensu.ca/~stat455/lecturenotes/set3.pdf

Answer 2

This is really basic Markov Chain Theory. You should refer to a standard textbook such James Norris's Markov Chains. I am going to refer your state A as state 1, and state B as state 2.

Let $P$ denote your transition matrix.

$P^n_{ij}$ gives $P(X_n=j|X_0=i)$

As $n\rightarrow\infty$, $P_{ij}\rightarrow\pi_j$, which is known as the stationary distribution. I think your simulation worked out $\pi_j$. This means, if your Markov Chain runs for a long time, your current location depend less and less on your initial position.

To solve this, you need to solve the simultaneous equation $\pi P = \pi$, where $\pi = (\pi_1,\pi_2)$.

Answer 3

This is called the stationary dstribution and solves $\pi=\pi P$. Thus, $\pi_A=\pi_AP_{AA}+\pi_BP_{BA}$ and $\pi_B=\pi_AP_{AB}+\pi_BP_{BB}$, which, in your case, yields $\pi_A=\frac58$ and $\pi_B=\frac38$.