Expected value of a random walk time

1.2k Views Asked by At

I've been doing some old competitive programming tasks and found one, for which I cannot find a general solution or a solving principle.

The math part of the question is:
You have N vertices, labeled $1$ to $N$. For each vertex $v$, you are given a neighbour $u$ and a probability $p_{vu}$, that you will visit $u$ from $v$. You finish this random walk when you reach $N$. Backwards loops are possible ($p_{vv}$ can be greater than $0$). Also, a probability of a $vu$ path can be $0$.

What is the expected time $T$ of this random walk, if one passage from $v$ to $u$ symbolises $1$ time unit? If reaching N is impossible, you stop and say it is impossible.

I am thinking of a backwards approach but don't know, how to find a general system of equations that solve this, that aren't case specific.

1

There are 1 best solutions below

2
On BEST ANSWER

You want to look at hitting times of Markov chains. You can check out Markov Chains by James Norris for an excellent account here. On that link, only the first chapter is available, but that's all you need. In particular, look at Section 1.3. (Note that these links are from James Norris' homepage, so are the genuine link. You may well be able to find the whole pdf elsewhere, or look in a library for the book.)

Other books on Markov chains will also be helpful, eg by Geoffrey Grimmett.

The method linked above gives a set of equations to solve, which may or may not be helpful since you're looking at it from a programming point of view. In general I expect this is pretty difficult to implement, however. If you have structure on the graph, then you can do some analytic stuff to reduce the equations, but for general Markov chains... going to be slow to solve computationally for large $N$ (at least methods that I know)