Let $\mathbf{X} = (X_0, X_1, ..., X_n)$ be a finite length walk on a Markov chain with initial distribution $\lambda$, transition matrix $P$, over the state space $\Omega = \{1, 2, ..., l\}$.
Suppose that we also associate weights, $w_1, w_2, ..., w_l$ with each state in the state space.
I am interested in calculating certain statistics arising from this set up.
For example, let $W$ denote the total weight of the walk (that is, the sum of the weights associated with each successive state that the walk lands on).
How can we calculate $\mathrm{Var}(W)$?
What can be said about the distribution of W?
Any insight, including approximations, (e.g. using CLT for distribution of $\overline{W}$) or any heuristic inference methods, would be much appreciated.
If it helps, the transition probabilities are expected to be small.
Additional info:
Let $W_k$ denote the weight associated with the $k$-th state, $X_k$, so that $W = W_0 + W_1 + \cdots W_n$.
Let $\mathbf{w} = \begin{bmatrix} w_1 & w_2 & \cdots & w_l \end{bmatrix}^T$.
It is relatively straightforward obtain the formulae, $$ \begin{align} \mathbb{E}(W) & = \sum_{k=0}^{n} \mathbb{E}(W_k) \\ & = \lambda \left( \sum_{k=0}^n P^k \right) \mathbf{w} \end{align} $$ and similarly, $$ \mathbb{E}({W_k}^2) = \lambda P^k \mathbf{w}^{\circ 2} $$ (where $\circ 2$ denotes the Hadamard power).
However, where I seem to be getting stuck is calculating $\mathbb{E}(W_k W_{k+r})$ (without getting a long winded formula which is essentially just the definition of expectation!). An expression for this should lead to an expression for $\mathrm{Var}(W)$...