Understanding proof that if $\rho_{xy} >0$ but $\rho_{yx}<1$, then x is transient.

560 Views Asked by At

Theorem:

If $\rho_{xy} >0$ but $\rho_{yx}<1$, then, x is transient.

Proof:

In general, let the Markov chain from state x to y be done so in K steps.

Then, $P^{K}_{xy}=P\left ( x,y_{1} \right )\cdot \cdot \cdot P(y_{K-1},y)>0$

From here, it is necessary that I show that the probability for the Markov chain at state x NOT to transit to state x at time of first return has a probability greater than $P^{K}_{xy}=P\left ( x,y_{1} \right )\cdot \cdot \cdot P(y_{K-1},y)>0$

I am unable to show this.

Any hints are appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

From the detail you give above I am not certain that I understand your question correctly, but I think the below interpretation seems plausible.

I am assuming that the notation you use that $\rho_{xy}$ is the probability that the hitting time at $y$ of the Markov chain started from $x$, is finite.

That is, if we define the first return time to a state $y$ of the Markov chain $X_n$ to be $$\tau_y^+ = \inf\{ n \geq 1 \, \colon \, X_n = y\},$$ then I will assume that $\rho_{xy}$ denotes the probability that the return time to $y$ is finite, starting from $x$ $$\rho_{xy} = \mathbf P_x[ \tau_y^+ < \infty].$$ For $x$ to be a transient state, we need to show that $\rho_{xx} < 1$.


To do this, we use the law of total probability and condition on whether or not the Markov chain hits $y$; we have \begin{align*} \rho_{xx} & = \mathbf{P}_x[ \tau_x^+ < \infty] \\ & = \mathbf{P}_x[ \tau_x^+ < \infty \, | \, \tau_y^+ < \infty ] \,\mathbf{P}_x[ \tau_y^+ < \infty] \, \, + \, \,\mathbf{P}_x[ \tau_x^+ < \infty \, | \,\tau_y^+ = \infty ] \,\mathbf{P}_x[ \tau_y^+ = \infty]\\ & = \mathbf{P}_x[ \tau_x^+ < \infty \, | \, \tau_y^+ < \infty ] \,\rho_{xy} \, \, + \, \,\mathbf{P}_x[ \tau_x^+ < \infty \, | \,\tau_y^+ = \infty ] \,(1 - \rho_{xy})\\ & \leq \mathbf{P}_x[ \tau_x^+ < \infty \, | \, \tau_y^+ < \infty ] \,\rho_{xy} \, \, + \, \,(1 - \rho_{xy}) \end{align*} where in the last line we simply used the fact that $0 \leq \mathbf{P}_x[ \tau_x^+ < \infty \, | \,\tau_y^+ = \infty ] \leq 1$. Rearranging the above, we have $$ \rho_{xx} \leq 1 - \rho_{xy}\bigg( 1 - \mathbf{P}_x[ \tau_x^+ < \infty \, | \, \tau_y^+ < \infty ] \bigg)$$ However, conditioned on the fact that the Markov chain visits $y$: the probability of returning to $x$ starting at $x$ is the same as the probability of returning to $x$ starting from $y$. That is $$ \mathbf{P}_x[ \tau_x^+ < \infty \, | \, \tau_y^+ < \infty ] = \mathbf{P}_y[ \tau_x^+ < \infty ] = \rho_{yx},$$ so we have $$ \rho_{xx} \leq 1 - \rho_{xy}( 1- \rho_{yx}).$$ When you combine this with the knowledge that $\rho_{xy} > 0$ and $\rho_{yx}<1$, it follows that $\rho_{xx} < 1$, and hence is transient.