Potential associated to a Random Walk on $\mathbb{Z}$

60 Views Asked by At

This is an example I found on Markov Chains, J.P.Norris: Consider the simple discrete-time random walk on $\mathbb{Z}$ with transition probabilities $p_{i-1,i} = q < p = p_{i,i+1}$. Let $c > 0$ and suppose that a cost $c_{i}$ is incurred every time the walk visits state $i$. What is the expected total cost $\phi_{0}$ incurred by the walk starting from $0$ where $$\phi_{0} = \mathbb{E}_{0} [ \sum_{n = 0}^{\infty} c(X_{n}) ]$$ Note that $$\phi_{0} = 1 + p \phi_{1} + q \phi_{-1} $$ and $$\phi_{i} = c \phi_{i-1}$$ so \begin{align*} \phi_{0} &= 1 + pc\,\phi_{0} + q\,\frac{\phi_{0}}{c} \\ &= 1 + \big(pc + \frac{q}{c} \big)\, \phi_{0} \end{align*} which leads to $$\phi_{0} = \frac{c}{c - p\,c^{2} -q}$$ At this point it's clear that if $\phi_{0}$ is seen as a function of c \begin{align*} \begin{cases} \phi_{0} &> 0 \quad \text{in $(\frac{q}{p},1)$} \\ \phi_{0} &< 0 \quad \text{in $(0,\frac{q}{p}) \cup (1,\infty)$} \end{cases} \end{align*} It's clear for me that $\phi_{0} = \infty$ if $c \geq 1$ because the random walk diverges to $\infty$ as $n \to \infty$ but I don't understand why $\phi_{0}$ also diverges if $c \in (0,\frac{q}{p})$, Can you help ?