Example of a Markov chain transition matrix that is not diagonalizable?

Question

It is well-known that every detailed-balance Markov chain has a diagonalizable transition matrix. I am looking for an example of a Markov chain whose transition matrix is not diagonalizable. That is:

Give a transition matrix $M$ such that there exists no invertible matrix $U$ with $U^{-1} M U$ a diagonal matrix.

Is there a combinatorial interpretation for the Jordan blocks that I can see directly from the graph?

Edit: I found this example here: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-262-discrete-stochastic-processes-spring-2011/video-lectures/lecture-8-markov-eigenvalues-and-eigenvectors/MIT6_262S11_lec08.pdf on page 21.

\begin{array}{ccc} 1/2 & 1/2 & 0\\ 0 & 1/2 & 1/2\\ 0 & 0 & 1 \end{array}

This particular example has two non-recurrent states, which is not really what I want. So I am modifying my question to ask for an example of a Markov chain for which every state is recurrent.

Answer 1

Consider the matrix, $$M={1\over300}\pmatrix{210&40&24\cr15&210&96\cr75&50&180\cr}$$ Note that I adopt the convention that the columns, not the rows, are to add up to $1$. Now $1/2$ is an eigenvalue, since the first row of $$M-{1\over2}I={1\over300}\pmatrix{60&40&24\cr15&60&96\cr75&50&30\cr}$$ is four-fifths of the 3rd row. But also $1$ is an eigenvalue, and the eigenvalues add up to $2$, so $1/2$ is a repeated eigenvalue. Its eigenspace is one-dimensional, since $M-(1/2)I$ has rank $2$, so $M$ is not diagonalizable.

EDIT. I thought I'd have a go at finding an example with smaller numbers. Let $$M={1\over5}\pmatrix{2&2&1\cr1&2&1\cr2&1&3\cr}$$ The eigenvalues are $1$ (since the matrix is column-stochastic), $1/5$ (since the 1st and 3rd columns of $$M-{1\over5}I={1\over5}\pmatrix{1&2&1\cr1&1&1\cr2&1&2\cr}$$ are identical), and $1/5$ again (since the eigenvalues add up to $(2+2+3)/5=7/5$). $M-(1/5)I$ has rank $2$, so the eigenspace of the eigenvalue $1/5$ is 1-dimensional, so $M$ is not diagonalizable.

Answer 2

In hindsight, it really isn't difficult to produce a nice looking non-diagonalisable Markov transition matrix that hasn't any transient states. Let $u^T=(1,1,1)$ and let $x$ be any positive probability vector. Then $x$ and $u^T$ can be completed to two invertible matrices $R=\pmatrix{u^T\\ v^T\\ w^T}$ and $L=\pmatrix{x&y&z}$ such that $LR=I$. Now let $$ M=L\pmatrix{1&0&0\\ 0&\lambda&0\\ 0&\mu&\lambda}R=xu^T+\lambda(yv^T+zw^T)+\mu zy^T. $$ Then $u$ is a left eigenvector of $M$. Since $xu^T>0$, we have $M>0$ when $\lambda$ and $\mu$ are sufficiently small. Moreover, $M$ is non-diagonalisable when $\mu\ne0$. It follows that $M$ is a transition matrix without any transient states if $\mu\ne0$ and $\lambda,\mu$ are sufficiently small.

In particular, if we put $\lambda=0$, it is easy to generate an $M$ that, up to a fraction, is a small integer matrix. For example, if we put $\lambda=0$, $$ R=\pmatrix{1&1&1\\ -1&2&-1\\ 2&-1&-1} \ \text{ and }L=R^{-1}=\frac13\pmatrix{1&0&1\\ 1&1&0\\ 1&-1&-1}, $$ then \begin{aligned} M_1&=L\pmatrix{1&0&0\\ 0&0&0\\ 0&\frac13&0}R =\frac{1}{9}\pmatrix{2&5&2\\ 3&3&3\\ 4&1&4},\\ M_2&=L\pmatrix{1&0&0\\ 0&0&0\\ 0&\frac14&0}R =\frac{1}{12}\pmatrix{3&6&3\\ 4&4&4\\ 5&2&5},\\ M_2&=L\pmatrix{1&0&0\\ 0&0&0\\ 0&\frac16&0}R =\frac{1}{18}\pmatrix{5&8&5\\ 6&6&6\\ 7&4&7} \end{aligned} are all nice-looking examples of $M$. $$ M_4=L\pmatrix{1&0&0\\ 0&0&0\\ 0&\frac12&0}R =\frac{1}{6}\pmatrix{1&4&1\\ 2&2&2\\ 3&0&3} $$ is also a valid example although it contains a zero entry.