Transition matrix and communicating classes

1.2k Views Asked by At

enter image description here

enter image description here

Firstly I wanted to check if they have a mistake in the solution. So for the transition matrix, the element p11 should be 0 and p12 should be 1, but they have it the other way round so I just wanted to check whether this was a mistake or not.

Secondly If a set is a communication class (CC) e.g {1,2,3} and the state {1} is also a CC then does that mean we don't need to include {1} separately as it has already been included in {1,2,3}? So basically if the communication classes were {1,2,3} and {1} then is that the same as writing just {1,2,3) as 1 has already been included in {1,2,3}

Thirdly is it a mistake again in the solutions when they have included {1} as a CC. Because it doesn't have a loop on itself so how can it be a CC.

Fourthly how would we determine if a CC set A is closed? I know how to determine if a state i that's a CC is closed for which we see if pii=1 but for a set A consisting of more than one state how do we determine if that set is closed? e.g if a CC {1,2} is closed then what do we need to check whether or not its closed?

Any help would be much appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

"Firstly I wanted to check if they have a mistake in the solution. So for the transition matrix, the element p11 should be 0 and p12 should be 1, but they have it the other way round so I just wanted to check whether this was a mistake or not."


You are correct. $p_{1,1}$ should be $0$ and $p_{1,2}$ should be $1$.


"Secondly If a set is a communication class (CC) e.g {1,2,3} and the state {1} is also a CC then does that mean we don't need to include {1} separately as it has already been included in {1,2,3}? So basically if the communication classes were {1,2,3} and {1} then is that the same as writing just {1,2,3) as 1 has already been included in {1,2,3}"

"Thirdly is it a mistake again in the solutions when they have included {1} as a CC. Because it doesn't have a loop on itself so how can it be a CC."


A state $i$ communicates with a state $j$ if both $i \rightarrow j$ and $j \rightarrow i$, i.e. you can get from one state to the other and vice versa.

Communication is an equivalence relation and the set of states that communicate forms equivalence classes.

Equivalence classes forms a partition of the set of states.

A partition of a set is a grouping of the set's elements into non-empty subsets, in such a way that every element is included in one and only one of the subsets.

So if a state $i$ communicate with state $j$, which can be written $i \leftrightarrow j$, then they belong to the same communicating class and they are related to each other $i \sim j$ and belong to the same equivalence class.

So $\{1,2,3\}$ and $\{1\}$ can not be communicating classes at the same time.


"Fourthly how would we determine if a CC set A is closed? I know how to determine if a state i that's a CC is closed for which we see if pii=1 but for a set A consisting of more than one state how do we determine if that set is closed? e.g if a CC {1,2} is closed then what do we need to check whether or not its closed?"


A communicating class is closed if the probability of leaving the class is zero. In your case consider the communicating class $\{4, 5\}$. Once you are there or if you start there you can never leave state $4$ and state $5$, i.e. its closed because the probability of leaving the class is zero.

And the same is true for $\{6\}$. Once you are there you can never leave.

If you only have the transition matrix and you want to figure out if a communicating class is closed or not you can look where the non-zero entries are in the matrix, e.g. if $\{4, 5\}$ is a CC like it is here then its closed if the entries that start with a $4$ or a $5$ and end with $1-3$ and $6$ are zero, i.e. the rows $4$ and $5$ should be zero except for the entries between state $4$ and $5$. $p_{4,5}$ and $p_{5,4}$ has to be non-zero because the states are communicating and $p_{4,4}$ and $p_{5,5}$ could be either non-zero or zero, it does not matter.