Covariance of states of a finite Markov chain

Question

I know it is possible to construct a covariance matrix for states of a Markov chain but I cannot seem to find a proper way to compute it. I will attach some theories I found from Kemeny and Snell's FMCs which states about limiting covariance.

I just cannot understand how the covariance matrix can be generated using this. I need to know whether there is a much easier way to approach this problem. In the same text in an example the have computed the convariance matrix without any steps showing how it was computed.

Hope someone could help me out on this and give me a better insight to the concept of covarince with respect to states in a Markov chain. Thanks !

Answer 1

Try this (R code) a <- A[1,] # Limiting matrix C <- mat.or.vec(m,m) # mxm covariance matrix delta <- function(i,j) {delta <- 0; if (i==j) delta <- 1; return(delta)} for (i in 1:m) { for (j in 1:m) { C[i,j] <- a[i]*Z[i,j]+a[j]*Z[j,i] - a[i]*a[j] - a[i]*delta(i,j) } } Result:

375*C [,1] [,2] [,3] [1,] 134 -18 -116 [2,] -18 36 -18 [3,] -116 -18 134 Kemeny's example on page 89.