I am trying to understand this following derivation from a book and need some help.
Suppose there is a homogeneous, irreducible, aperiodic Markov chain, whose state at time $t$ is $C_t$, and can take the value $\mathbf v = (1, 2, ..., m)$. Also define $\mathbf V = diag(1, 2, ..., m)$.
Let $\delta$ be the stationary distribution of $C_t$, i.e. $Pr(C_t = i) = \delta_i$.
Let $\mathbf\Gamma$ be the transition matrix of the said Markov chain. Assume $\mathbf \Gamma$ is diagonalizable, i.e. $\mathbf \Gamma$ = $\mathbf {U\Omega U^{-1} }$, where $\mathbf \Omega = diag(1, \omega_2, \omega_3, ..., \omega_m)$ contains the eigenvalues, and $\mathbf U$ contains the right eigenvectors of $\mathbf \Gamma$.
Naturally, $\mathbf \Gamma$ has the properties $\mathbf{\delta \Gamma = \delta}$ and $\mathbf { \Gamma 1' = 1' } $.
The book then proceeds to derive a formula for the lagged covariance of the state of the Markov chain:
$Cov(C_t, C_{t+k}) = \mathbf{ \delta V U \Omega^k U^{-1} v' - (\delta v')^2 }$ $ = \mathbf{ a \Omega^k b } - a_1 b_1$ $ = \sum_{i=2}^m a_i b_i \omega_i^k$
where $\mathbf{ a = \delta V U }$ and $\mathbf{ b' = U^{-1} v' }$.
I am able to derive the first step:
$ Cov (C_t, C_{t+k}) = E(C_t * C_{t+k}) - E(C_t)^2 $
$E(C_t) = \sum_{i=1}^m \delta_i * i = \delta \mathbf{v'}$
$E(C_t * C_{t+k}) = \sum_{j=1}^m \sum_{i=1}^m Pr(C_{t+k} = j, C_t = i) * i * j = \sum_{j=1}^m \sum_{i=1}^m (\Gamma^k)_{ij} * \delta_i * i * j = \delta \mathbf{ V \Gamma^k v'} = \delta \mathbf{ V U \Omega^k U^{-1} v'} $
So $Cov (C_t, C_{t+k}) = \delta \mathbf{ V U \Omega^k U^{-1} v'} - (\delta \mathbf{v'})^2$
I can also see that because the first column of $\mathbf U$ must be $\mathbf 1'$, so that $a_1 = \mathbf{\delta V U_1 = \delta V 1' = \delta v'} $.
However, why is $b_1 = \mathbf{ \delta v' }$?
Ah ... I am stupid. Now I see the answer. I am keeping the question in case someone later has the same question.
Transpose the $\mathbf \Gamma$ matrix, then we have a diagonalization for $\mathbf \Gamma'$:
$\mathbf{ \Gamma' = (U^{-1})' \Omega U'} $
Also, we have: $\mathbf{ \Gamma' \delta' = \delta' }$. Therefore, $\delta'$ is an eigenvector of $\mathbf \Gamma'$ with the eigenvalue 1. Therefore, $\mathbf{ \delta' = (U^{-1})'_1 }$, the first column of $\mathbf{ (U^{-1})' }$. Therefore, $\delta$ is the first row of $\mathbf U^{-1}$. So $b_1 = \delta \mathbf v'$.