Finding positive recurrent states in a markov chain

Question

I am given the transition matrix above in a Markov Chain with states $\{1,2,3,4,5,6\}$, and I am asked to find the stationary distribution of this chain. As I noticed state 4 was transient, I tried to find the value of the stationary distribution for the rest of states ( by simply solving a system of equations)

Now as we received the solutions, the first 4 states had no stationary distribution, for the first 3 were null recurrent and the 4th transient. Only states 5 and 6 were positive recurrent and had a value greater than zero.

My question is, how can I intuitively recognize positive and null recurrent states by looking at a matrix ? I had thought the first 3 states to form an irreducible closed set, and thus be positive recurrent. I still do not understand why they aren't, and how I can figure whether they have a stationary distribution or not.

Thank you

Answer 1

Your transition matrix is incorrect because elements of the 3rd row do not sum to $1.$ That may explains your confusing results. Please edit your Question accordingly.

If the second row is changed to $(1/2, 1/4, 1/4, 0,0, 0),$ then you are correct that $\{1,2,3\}$ forms a persistent class that is an ergodic chain. (Obviously ergodic because the square of the transition matrix has all positive elements.)

In R code, its transition matrix is shown below along with a method for finding its stationary distribution.

P = (1/12)*matrix(c(4,4,4, 
                    3,9,0, 
                    6,3,3), nrow=3, byrow=T);  P
          [,1]      [,2]      [,3]
[1,] 0.3333333 0.3333333 0.3333333
[2,] 0.2500000 0.7500000 0.0000000
[3,] 0.5000000 0.2500000 0.2500000
rowSums(P)
[1] 1 1 1                    # verify that rows sum to 1
g = eigen(t(P))$vector[,1]   # left eigenvect w/ least modulus    
g = as.numeric(g/sum(g));  g # normed stationary distn
[1] 0.3103448 0.5517241 0.1379310
g %*% P                      # verify stationarity
          [,1]      [,2]     [,3]
[1,] 0.3103448 0.5517241 0.137931

Thus the stationary (and limiting) distribution is $(0.3103448, 0.5517241, 0.137931).$

Notes: R finds right eigenvectors, so we transpose P. Some eigenvectors (other than the one we want) may have complex values, so as.numberic removes useless complex-number notation. Dividing g by the sum of its elements ensures that g is a probability distribution. The code %*% is for matrix multiplication.