Limiting state vector of a $3$-state Markov chain

Question

Given a $3 \times 3$ Markov chain matrix, calculate its limiting state vector $\bf v$. Is this vector $\bf v$ a row vector or a column vector?

---

Does anyone know what is meant by a limiting state vector? I've googled around and can't any find information on a it, only a limiting matrix. Are they the same thing?

Answer 1

Whether or not it's a row or column vector depends on your conventions, and for transition matrices I've seen both conventions. But if you represent distributions over states by column vectors (so that if $\mathbf x$ is the distribution at one step of the Markov chain, and $P$ is the transition matrix, then $P \mathbf x$ is the distribution at the next step) then the limiting state vector is also a column vector.

The limiting state vector is the vector of probabilities for the limiting distribution of the Markov chain, assuming it exists: the distribution over states you converge to after many steps. This is always a stationary distribution (it satisfies $P\mathbf x = \mathbf x$ and $x_1 + \dots + x_n = 1$) but on the other hand:

• If the Markov chain is not irreducible - there are multiple non-interacting components - there are multiple stationary distributions, and the limiting distribution depends on the starting state;
• If the Markov chain is not aperiodic - the long-term distribution of the state depends on the number of steps taken modulo $k$ for some $k$, due to going around in some cycle - there is no limiting distribution, although there may be a stationary distribution.