I have 3 little questions regardings invariant and stationary probability distributions.
Let $E=\{a,b\}$ be a state space and $\textbf{Q}$ a transition matrix $$\textbf{Q}:=\begin{pmatrix}1-p&p \\ q &1-q\end{pmatrix}$$ with $0<p,q \leq 1$ and $p+q<2$.
Then a measure $\mu$ is invariant if $\mu \textbf{Q}=\mu$. Now this means the following must hold $$\begin{cases}\mu(a)+\mu(b)=1\\ p\cdot\mu(a)-q\cdot\mu(b)=0\end{cases}$$
- Now I don't understand why $p\cdot\mu(a)-q\cdot\mu(b)=0$ must hold?
(If I have $3$ states would it be $p\cdot\mu(a)-q\cdot\mu(b) -0\cdot\mu(c)=0$?)
The invariant distribution is $$\mu(a)=\frac{q}{p+q}\quad \mu(b)=\frac{p}{p+q}$$ We have that $$\lim\limits_{n\to \infty}\textbf{Q}^n=\frac{1}{p+q}\begin{pmatrix}q&p \\ q &p\end{pmatrix}$$ So $\lim\limits_{n\to \infty}\textbf{Q}^n=(\mu,\mu)^T$
So the transition matrix is converging to the invariant probability distribution. Is the invariant probability always the stationary distribution (and vice versa)? It confuses me that we have invariant/stationary distributions defined as vectors, but now the stationary distribution is the matrix?
Does the limit matrix $\lim\limits_{n\to \infty}\textbf{Q}^n$ always have the same row? Meaning the first row of the limit matrix is also the second etc. (Here it is $\mu$ and again $\mu$)
Let $P(i\mid j)$ denote the conditional probability that the next state of the system will be $i$ given that it is in state $j$. With this the state transition matrix is $$\begin{bmatrix} P(a\mid a)& P(b\mid b)\\ P(a\mid b)& P(b\mid b) \end{bmatrix}=\begin{bmatrix} 1-p& p\\ q& 1-q \end{bmatrix}.$$
Given the state transition matrix and assuming that the probabilities that the system is in state $a$ or $b$ are $\mu(a)$ and $\mu(b)$, respectively, one can calculate the probability that the system willy in state $a$ is
$$\mu'(a)=P(a\mid a)\mu(a)+p(a\mid b)\mu(b)$$
The same way
$$\mu'(b)=P(b\mid a)\mu(a)+P(b\mid b)\mu(b).$$
If $\mu'(a)=\mu(a)$ and $\mu'(b)=\mu(b)$ then the these probabilities are the stationary state probabilities.
So, if one wants to calculate the stationary probabilities then the one has the following three equations for $\mu(a)$ and $$\mu(b)$::
$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$ $$\mu(b)=p\mu(a)+(1-q)\mu(b).$$ $$\mu(a)+\mu(b)=1.$$
Too many equations... But if the stationary probabilities exist then it is enough to take only two of the equations above:
$$\mu(a)=(1-p)\mu(a)+q\mu(b)$$ $$\mu(a)+\mu(b)=1.\tag 1$$
From the first equation you get
$$0=q\mu(b)-p\mu(a).$$
This answers your first question.
The $n^{th}$ power of the state transition matrix is called the $n$ step transition matrix because it shows the state transition probabilities (in one) if we let the chain step ahead $n$ times.
If $p,q>0$ then the limit of the $n^{th}$ power of the state transition matrix tends to a matrix whose both rows equal the solutions (vector) of $(1)$. If you apply this limiting matrix to the vector of the stationary distributions then you get them back:
$$\frac1{p+q}[q \ p] \frac1{p+q}\begin{bmatrix} q&p\\ q& p \end{bmatrix}=\frac1{(p+q)^2} [q^2+pq\ \ qp+p^2] =\frac1{p+q}[ q \ p].$$
I expect that this latter relationship between the limit of the $n$-step transition matrix and the stationary solution enlightens you.