Expected time for a Markov chain to reach a state

50 Views Asked by At

Assume we have a bag containing balls. At each time step, a ball is added with probability $p$ or a ball is removed with probability $1-p>p$. If the bag is empty, it remains so with probability $1-p$ or a ball is added with probability $p$. So that, the sequence of number of balls in the bag at each time $n \in \mathbb{N}$ forms an irreductible (aperiodic) Markov chain:

\begin{align} & X_{n+1} = X_n + \mathbb{1}_{ \{ W_{n+1}=H \} } - \mathbb{1}_{ \{ W_{n+1}=T \} }, & \text{ if } X_n>0, \\ & X_{n+1} = \mathbb{1}_{ \{ W_{n+1}=H \} }, & \text{ if } X_n=0, \\ & X_0 \in \mathbb{N} \end{align}

with $(W_n)_{n \in \mathbb{N}}$ being a sequence of indedepent random variables such that, for all time step $n$, $\mathbb{P}(W_n=H)=1-\mathbb{P}(W_n=H)=p$.

I am interesting in computing the mean time $T^0$ to empty the bag given the initial number of balls is $1$: $X_0=1$ a.s.; I wrote \begin{equation} \mathbb{E}_1[T^0] = \sum_{n \in \mathbb{N}} n \mathbb{P}_1 (T^0 = n). \end{equation} The event ${T^0=n}$, given ${X_0=1}$ corresponds (unless I am mistaken) to the event that the number of balls is increased by 1 $k$ times and decreased (by one) $k+1$ times, for $k \in \mathbb{N}$ such that $k+k+1=1+2k=n$, but the order matters (we should start with (at least) one increase of $1$ ball). All the paths going from state $X_0=1$ to $X_n=0$ should thus have same probability $p^k (1-p)^{k+1}$ and $$ \mathbb{P}_1 (T^0 = n) = \#\{ \text{Number of possible paths from $1$ to $0$ that reach state $0$ exactly at time $n$ } \} \times p^k (1-p)^{k+1}. $$

I have two questions then: i) What would be the (cardinal) number of such paths? ii) Is there a more "direct" way to compute the mean time in question using a general result? For instance, if interested in computed the average mean return time to state $0$, the invariant probability measure $\pi$ would be useful as $$ \mathbb{E}_0[T^0]=1/\pi(0). $$

Thank you!