proof of the existence of a stationary distribution in a Markov chain

238 Views Asked by At

I'm going through the following proof and a few points are not necessarily obvious to me, i would love to fully understand this proof so i would appreciate explanations.

enter image description here

Firstly should we note require that $\sum_{i=1}^{n} v_i=1$ for $v_i \in v$ since it is a probability distribution? Or is this solved by the fact once we have a $v$ then we know if its values are positive we can just normalise it by the sum?

Secondly why do we require the lazy chain $q$? I understand we obtain $r$ from it but why do we need $r=q^{k-1}$ It seems r=p would work for the proof would it not?

1

There are 1 best solutions below

0
On BEST ANSWER

The eigenvector can always just be normalized into a probability distribution as long as its nonzero entries have the same sign. That's why you need to check that the entries all have the same sign in the first place. Just in general, these linear-algebraic style proofs don't really need to work with a probability distribution all the way through, they can work with vectors and then convert them to probability distributions when needed/possible.

The lazy chain is an aperiodic chain with the same stationary distribution as the given chain. (Honestly, if I were writing a passage like that for pedagogical purposes, I would spell this point out, because there's nothing special about this particular lazy chain. But I digress.) Irreducibility and aperiodicity together get you to $r(x,y)>0$, which is the key fact you need to carry out the proof.

If you didn't make the lazy chain, you would run into trouble when $p$ is periodic, because in this case you wouldn't have $p^{k-1}(x,y)>0$ for all $x,y$. In particular, if the period doesn't divide evenly into $k-1$ (e.g. if $p=\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$) then $p^{k-1}(x,x)$ will be zero for every $x$.