Weighted Walk on Markov Chain

156 Views Asked by At

Let $\mathbf{X} = (X_0, X_1, ..., X_N)$ be a finite walk on a Markov chain, with transition matrix $P$, over the state space $\Omega = \{1, 2, ..., l\}$, starting at state $1$ and ending at state $l$.

Note that $N$ is not fixed - the walk terminates when it reaches state $l$.

We also assume that $P$, together with an extra transition from state $l$ to state $1$ with probability $1$ is irreducible (i.e. there is a path from state $1$ to state $l$ which goes through state $k$ for every $ 1 \le k \le l$).

Suppose that we also associate weights, $w_1, w_2, ..., w_l$ with each state in the state space.

I am interested in calculating certain statistics arising from this set up.

For example, let $W$ denote the total weight of the walk (that is, the sum of the weights associated with each successive state that the walk lands on).

How can we calculate $\mathrm{Var}(W)$?

What can be said about the distribution of W?

Any insight, including approximations, (e.g. using CLT for distribution of $\overline{W}$) or any heuristic inference methods, would be much appreciated.

If it helps, the transition probabilities are expected to be small.


Additional info:

Note that this differs from this similar question in that the length of the Markov chain is not fixed. This represents a significant difference in the statistic, $W$, used in each question.