Proofs with the Markov property

282 Views Asked by At

I'm struggling with the following question, which I think involves the Markov property:

Consider the simple random walk with parameter $0 < p < 1$. This is a Markov chain on state space $\mathbb{Z}$ with transition probabilities $p(x, x+1) = p$ and $p(x, x-1) = 1 -p$. Let $q(x, y) = P_{x}(T_{y} < \infty)$ (i.e. the probability that time of first visit to $y$, starting in state $x$, is finite).

Prove $q(2, 0) = q(2, 1) \cdot q(1, 0)$.


This intuitively makes sense to me because to go to from State $2$ to State $0$, you always need to pass through State $1$. I have no idea how to prove it formally though. This is an exercise from my book, and I am really new to Markov chains, so I would greatly appreciate any help.

1

There are 1 best solutions below

0
On

The rigorous proof of this result depends on the strong Markov property of random walk on $\mathbb{Z}$, it goes as follows:
Let $S_n := X_0+\sum_{k=1}^n X_k$ be a random walk on $\mathbb{Z}$, so if we start from $S_0 = 2$ and it can reach the point $x = 0$, then it must pass through point $x=1$ earlier some time. Therefore \begin{align} q(2, 0) = P_2(T_0<\infty) &= \mathbb{E}_2\left[\mathbf{1}_{\{T_0<\infty\}}\right] \\ &= \mathbb{E}_2\left[\mathbf{1}_{\{T_0<\infty\}}\mathbf{1}_{\{T_1<\infty\}}\right] \end{align} On $\{T_1=\infty\}$, we know LHS=RHS=$0$, so we only need to prove this result on $\{T_1<\infty\}$. If we condition the expression above on the $\sigma$-algebra $\mathscr{F}_{T_1}$, then
\begin{align} q(2, 0) &= \mathbb{E}_2\left[\mathbb{E}_2\left[\mathbf{1}_{\{T_0<\infty\}}\mathbf{1}_{\{T_1<\infty\}}|\mathscr{F}_{T_1}\right]\right] \\ \text{Since }\{T_1<\infty\}\in\mathscr{F}_{T_1} \ \Rightarrow\ \ &=\mathbb{E}_2\left[\mathbf{1}_{\{T_1<\infty\}}\mathbb{E}_2\left[\mathbf{1}_{\{T_0<\infty\}}|\mathscr{F}_{T_1}\right]\right] \\ \text{By strong Markov property} \ \Rightarrow \ \ &=\mathbb{E}_2\left[\mathbf{1}_{\{T_1<\infty\}}\mathbb{E}\left[\mathbf{1}_{\{T_0<\infty\}}|S_0=S_{T_1}\right]\right] \\ T_1<\infty \ \Rightarrow \ \ &=\mathbb{E}_2\left[\mathbf{1}_{\{T_1<\infty\}}\mathbb{E}\left[\mathbf{1}_{\{T_0<\infty\}}|S_0=1\right]\right] \\ &=\mathbb{E}_2\left[\mathbf{1}_{\{T_1<\infty\}}P_1(T_0<\infty)\right] \\ &=\mathbb{E}_2\left[\mathbf{1}_{\{T_1<\infty\}}\right] P_1(T_0<\infty)\\ &=q(2,1)q(1,0) \end{align}