Intuition / Interpretation of Balance Equations (Markov Chains)

433 Views Asked by At

I'm currently studying a module on Stochastic Processes and I'm quite confused as to the interpretation of Global/Local Balanced Equations for Discrete Markov Chains.

From what I've learnt, Global balanced equations satisfy $$ \pi = \pi\mathbf{P} $$ where $\pi$ is the distribution and $ \mathbf{P} $ is the transition probability matrix. Element-wise, it is $$ \pi(j) = \sum_{k\in S} \pi(k)P_{kj} $$ or $$ P(X_n = j) = \sum_{k \in S}P(X_n = k)P(X_n+1 = j | X_n = k) = P(X_{n+1} = j) \tag{*}$$

I understand the above equation from the context of regular Markov chains and the limiting distribution, but there's also an explanation that the LHS of the equation is the probability of leaving node $ j$ to other nodes $ i \neq j $, and the RHS is the probability that nodes $ i \neq j $ changes to node $j$. However, I cannot link the equation $(*)$ to this intuition.

Also, for local balanced equations, where $$ \pi(i)P_{ij} = \pi(j)P_{ji} $$ What is the interpretation of the above? Is it that the probability from state $j$ to $ i = $ probability from state $ i $ to $j$? How do I interpret it with the equation?

1

There are 1 best solutions below

1
On BEST ANSWER

Equation $(*)$ is just a special case of the law of total probability: whenever $B_1, B_2, \dots, B_k$ are a partition of the sample space (that is, whenever exactly one of them always holds) and $A$ is any other event, $$ \Pr[A] = \sum_{i=1}^k \Pr[A \mid B_i] \Pr[B_i]. $$ Here, $B_i$ is the event "$X_n = i$" and $A$ is the event "$X_{n+1}=j$".


The local balance equations (or detailed balance equations, or reversibility equations) are not guaranteed to hold for all Markov chains. In general, they hold for those Markov chains which can be interpreted as random walks on weighted, undirected graphs.

If the equation $\pi(i) P_{ij} = \pi(j) P_{ji}$ holds for all $j$, then we can sum over all $j \in S$ to get $$ \sum_{j \in S} \pi(i) P_{ij} = \sum_{j \in S} \pi(j) P_{ji} \implies \pi(i) = \sum_{j \in S} \pi(j) P_{ji} $$ which is just the global condition.

In practice, the local balance equations are much easier to solve, when a solution to them exists. So we often try solving them, and if it works, we're happy - without thinking about why a particular Markov chain does or does not have a reason to satisfy them.

One nice special case where we do have an intuitive reason to expect reversibility is when we are doing a random walk on the number line. Here, $S$ is a finite or infinite interval of integers, and $P_{ij} = 0$ if $|i-j| > 1$: from each state, we can only go to the next integer, or the previous integer, or stay put.

Suppose that we take $N$ steps in this random walk, where $N$ is very large. We expect to see state $i$ about $N \pi(i)$ times, and so we expect to take a step from $i$ to $i+1$ about $N \pi(i) P_{i,i+1}$ times. Similarly, we expect to take a step from $i+1$ to $i$ about $N \pi(i+1) P_{i+1,i}$ times. However, such steps must always alternate: once we go from $i$ to $i+1$, we must go from $i+1$ back to $i$ before we go from $i$ to $i+1$ again! So $N \pi(i) P_{i,i+1}$ and $N\pi(i+1) P_{i+1,i}$ should be off by at most $1$, which motivates the condition $\pi(i) P_{i,i+1} = \pi(i+1) P_{i+1,i}$.