Measuring incoming communication in a Markov Model

44 Views Asked by At

Given a standard Markov Chain on discrete time and finite statespace, represented by a matrix $M$, with $\sum_{j=1}^d m_{ij}=1$.

I have a certain absorbing state k, where the incoming communication is very interesting, so i wish to measure that somehow . Calculating $(I-T)^{-1}$ is not possible due to numerical issues.

Until now i have just calculated $e_n=\sum_{i=1}^d m^n_{ik}-m^n_{kk}$. Where $m^n_{ij}$ is the i,j entry in $M^n$ for different $n$ values.

For example we could have $$M=\left(\begin{array}{c} 0.5 & 0.3 & 0.2 \\0.1 & 0.4 & 0.5 \\ 0 & 0 & 1 \end{array}\right).$$ where we are interested in the communication to state 3. (The real matrix is very large).

We would get $e_1=0.2+0.5=0.7.$

My question is: Is there a better way to describe incoming communication to a state, than the column sum? If not, how do i interpret it then? If not, what can you suggest?

1

There are 1 best solutions below

2
On BEST ANSWER

Based on your attempt, you seem to want to know the probability that you arrive to state $3$ from either state $2$ or state $1$.

Since this is an absorbing chain, you will need to break out the transient portion $M'$ from the absorbing portion:

$$M':=\left(\begin{array}{c} 0.5/.8 & 0.3/.8\\ 0.1/.5 & 0.4/.5 \end{array}\right) = \left(\begin{array}{c} .625 & .375\\ .2 & .8 \end{array}\right)$$

$M'$ gives the behavior conditional on the chain not entering the absorbing state.

Now, conditional on the chain not hitting the absorbing state, the long-run frequency that it is in state 1 and 2 (i.e., $\pi_1,\pi_2$) can be calculated by the usual Markov Chain equations (noting that $\pi_2=1-\pi_1$):

$$.625\pi_1+.2(1-\pi_1)=\pi_1\implies \pi_1=\frac{1}{2.875}\approx 0.35 \implies \pi_2 \approx 0.65$$

Now, the probability that we enter the absorbing state from each of these states is a conditional probability:

$$\mathrm{From State 1 :}=\frac{\pi_1\times m_{13}}{\pi_1\times m_{13} + \pi_2\times m_{23}}\approx\frac{.35\times .2}{.35\times.2+.65\times 0.5}=\frac{0.07}{0.395}\approx 0.17 \implies \mathrm{From State 2}=1-0.17=0.83$$

So, your absorber is apporox. $5\times$ more likely to get a signal from state 2 than from state 1