First hitting time expectation and Markov property

643 Views Asked by At

Let $H_A$ be the first hitting time, such that $H_A\geqslant1$, so we have $X_0=i\notin A$. All texts I looked at, state without any further justification that $$ \mathbb E(H_A\mid X_1=j, X_0=i)=1+\mathbb E(H_A\mid X_0=j). $$ follows from the Markov property.

It makes intuitive sense yet how does one derive it?

1

There are 1 best solutions below

0
On BEST ANSWER

Write $$H_A=\sum_{n=0}^\infty \prod_{k=0}^n 1_{(X_k\notin A)}=1_{(X_0\notin A)}\left[1+\sum_{n=1}^\infty \prod_{k=1}^n1_{(X_k\notin A)}\right]=1_{(X_0\notin A)}[1+H_A^\prime],$$ where $H_A=f(X_0,X_1,X_2,\dots)$ and $H_A^\prime=f(X_1,X_2,X_3,\dots)$. Therefore
\begin{eqnarray*} \mathbb{E}(H_A\mid X_1=j,X_0=i) &=&1+\mathbb{E}\left(H_A^\prime \mid X_1=j,X_0=i\right)\\[5pt] &=&1+\mathbb{E}\left(H_A^\prime \mid X_1=j\right) \\[5pt] &=&1+\mathbb{E}\left(H_A \mid X_0=j\right). \end{eqnarray*} Try to find the places in this argument where we use $i\notin A$, the Markov property, and the time homogeneity of the chain.