Is the highest probability path is always the most direct in this Markov process?

143 Views Asked by At

Markov Process

The state $A_{t+1}$ at time $t+1$ is distributed as Binomial($N$,$\frac{A_{t}}{N}$). In this Markov process there are therefore two absorbing states $0$ and $N$.

Question

To go from one state (say $A_0=68$) to another (say $A_t=70$), there is an infinite possible number of paths, each of them are associated with a certain probability.

  • Is the highest probability path always the one-step path?
  • Is there a rule to determine whether the highest probability path is the one-step path?

Answering the first question through computational methods

For $N=1000$, $A_0=350$ and $A_{end}=800$. The following is coded in R

A0 = 350
Aend = 800
N = 1000

reps = 1e5 # This is the number of replicates to draw the histogra. 'reps = 1e5' will take a bit long. Might be better to use '1e4' if you are willing to reproduce.
lens = numeric(reps)

for (rep in 1:reps)
{
    if (rep %% 100 == 0) cat(paste0(rep/reps," %\n"))
    A = A0
    len = 0
    while(TRUE)
    {
        A = rbinom(1,N,A/N)
        len = len + 1
        if (A == Aend) 
        {
            lens[rep] = len         
            break
        }
        if (A == 0 | A == N)
        {
            lens[rep] = NA
            break
        }
    }
}

hist(lens,breaks=50)

enter image description here

t = table(lens)
print(paste("The most often used path length is",names(t)[t==max(t)]))

The most often used path length is 428.

Note that I haven't checked for potential biased due to round off error but I would tend to think the result is trustful.