Reversible probability for a Markov Chain.

274 Views Asked by At

Considering the price of a share $X_n$ that evolves every $n$ day that increases of one euro with probability $0<p<1$ or decreases of one euro of probability $q=1-p$.

Assume that $X_0=10$ and $p<q$

I already proved that $(X_n)_{n\ge 0}$ it's a homogeneous Markov Chain with transition matrix: $$\begin{bmatrix} q & p & 0 &0& \dots &0& 0 \\ q & 0 & p &0& \dots &0& 0 \\ 0 & q & 0 &p& \dots &0& 0 \\ \\ 0 & 0 & 0& \dots&q &0 &p \end{bmatrix}.$$

The next question is to show that $(X_n)$ admits a probability reversible $\pi$. $\pi$ is reversible if and only if $$\forall x,y\in [0,N]: \pi(x)P(x,y)=\pi(y)P(y,x).$$

So here it's equivalent to $\pi(x)P(x,x+1)=\pi(x+1)P(x+1,x)$ which can be written as $$p\pi(x)=(1-p)\pi(x+1).$$

I am not sure how can I continue.

2

There are 2 best solutions below

3
On BEST ANSWER

This is a birth-death Markov chain, so $$\nu_n = \left(\frac p{1-p}\right)^n $$ is an invariant measure for $P$, that is, $\nu P=\nu$. To normalize $\nu$, we compute $$\sum_{n=0}^\infty \nu_n = \sum_{n=0}^\infty\left(\frac p{1-p}\right)^n = \frac{1-p}{1-2p}. $$ Set $$\pi_n = \frac{\nu_n}{\sum_{k=0}^\infty \nu_k} = \left(\frac p{1-p}\right)^n\left(\frac{1-2p}{1-p}\right) = \left(\frac p{1-p}\right)^n\left(1-\frac p{1-p}\right). $$ Then $\pi$ is a stationary distribution for $P$ as $\pi P = \pi$ and $\sum_{n=0}^\infty \pi_n=1$. We have $\pi_0=1-\frac p{1-p}$ and $$\pi_{n+1} = \left(\frac p{1-p}\right)^{n+1}\pi_0,$$ so $\pi$ satisfies the recurrence $$p\pi_n = (1-p)\pi_{n+1}, $$ as desired.

If we let $P^*_{ij} = \frac{\pi_j}{\pi_i}P_{ji}$, then $P^*$ is the time reversal of $P$ with respect to $\pi$, having transition probabilities $$ P^*_{ij} = \begin{cases} p, & i=j=0\\ p, & j=i-1\\ 1-p, & j=i+1. \end{cases} $$

0
On

Suppose you have a positive value for $\pi(0)$, equal to $\pi_0$. Then you have the recurrence relation

$$\pi(x+1)=\frac{p}{1-p} \pi(x),\pi(0)=\pi_0.$$

This recurrence can be solved explicitly: you get $\pi(x)=\left ( \frac{p}{1-p} \right )^x \pi_0$. Now normalize that.