Showing that $p_{k}=\frac{k}{k-1}p_{k-1}$ forall $k$ implies $p_{k}=\frac{k}{n}$

193 Views Asked by At

Suppose I have a Markov process over $n+1$ states $s_0,\ldots,s_n$. For $k\notin\{0,n\}$ the transition probabilities from $s_k$ are $\frac{1}{2}$ to both $s_{k-1}$ and $s_{k+1}$ and for $k\in\{0,n\}$ there is probability 1 of staying in place (absorbing states).

Denote by $p_k$ for $0<k<n$ (the transient states) the probability of ending up in $s_n$ when starting at $s_k$. Iv'e managed to show that for all such $k$

$$p_{k}=\frac{k}{k-1}p_{k-1}$$

And now need to show that this implies that $p_{k}=\frac{k}{n}$. This seemed easy enough (the question was phrased as "conclude that $p_{k}=\frac{k}{n}$") but for some reason I'm not able to. So far I only got

$$p_{k}=\frac{k}{k-1}p_{k-1}=\frac{k}{k-1}\frac{k-1}{k-2}p_{k-1}=\ldots=\frac{k}{k-1}\frac{k-1}{k-2}\frac{k-2}{k-3}\ldots\frac{2}{1}p_{1}=kp_{1}$$

I also showed, earlier in the same question, that the stationary distribution vector of this process has to be of the form $\pi = \alpha e_0 + (1-\alpha)e_n$ for some $\alpha\in [0, 1]$, a fact which may or may not be relevant.

2

There are 2 best solutions below

0
On BEST ANSWER

This is the symmetric gambler's ruin problem and there are many ways to solve this. (If you know anything about Renewal Theory, applying the Wald Equation, which is also a very simple martingale, is my vote.)

A nice finish that makes use of your existing work, i.e. with knowledge that

$p_{k}=\frac{k}{k-1}p_{k-1}=\frac{k}{k-1}\frac{k-1}{k-2}p_{k-1}=\ldots=\frac{k}{k-1}\frac{k-1}{k-2}\frac{k-2}{k-3}\ldots\frac{2}{1}p_{1}=kp_{1}$

is to assign $p_1 = c$ so $p_k =c\cdot k$ and determine $c$ via a suitable boundary condition.

Easy finish:
supposing your gambler starts in transient state $k\in\{1,2,..., n-1\}$, you can make use of 'first step analysis'. That is the probability of being absorbed in state $n$ is $p_k$ but if we condition of the first 'step' we have (this is total probability, or total expectation if you use indicators)

$p_k = \frac{1}{2}p_{k-1} + \frac{1}{2}p_{k+1}$
or
$p_{k+1} = 2\cdot p_k- p_{k-1}$
(note: solving this recurrence directly is yet another way to get the solution to Gambler's Ruin.)

Now select $k:= n-1$ and this reads
$1 = p_n= 2 p_{n-1}- p_{n-2} = 2 \cdot c \cdot (n-1) - c \cdot (n-2) = c\cdot\big(2n-2 -(n-2)\big) = c\cdot \big(n\big)\longrightarrow c = \frac{1}{n}$
which is the boundary information you needed.

2
On

Let $\{X_n:n=0,1,\ldots\}$ be a Markov chain on $\{0,1,\ldots,n\}$ with your given transition probabilities. Then in fact $X_n$ is a martingale, for clearly $$ \mathbb P(X_n\in\{0,1,\ldots,n\})=1 \implies \mathbb E[|X_n|]<\infty $$ and $$ \mathbb E[X_{n+1}\mid X_n] = \begin{cases}s_n,& X_n=s_n\\ s_0,& X_n=s_0\\ \frac12(X_n+1) + \frac12(X_n-1) = X_n,& X_n\notin\{s_0,s_k\} \end{cases}. $$ See my answer here for showing that $p_k = \frac kn$, $k=1,\ldots,n-1$ when $X_n$ is a martingale: Markov Chain Martingales Dembo 6.1.18