Markov chains example

Question

Your exam could be marked with a range of possible grades, simplified as on the following state diagram:

To begin with the chances are that you will pass with a standard result. Each 45 minutes (the time allocated for each answer) your grade could go up or down to a neighbouring grade or stay put.

There is a chance that your Genius is so great that the examiner cannot appreciate what you are saying and incorrectly marks it as nonsense.

What is your grade most likely to be after the 3 hour exam? What (if any) are the steady state results for a group of 14 students?

OK, I have done following:

Made transitional matrix:

 1/2     1/2      0    0
1/10    3/10    3/5    0
   0     1/2    2/5 1/10
1/10       0    4/5 1/10

Then computed P^4 since in 3 hours there is 4x45 mins

    7/50     401/1000       21/50     39/1000
    11/125   999/2500   1149/2500      33/625
    407/5000 779/2000   2383/5000      21/400
    417/5000 161/400     459/1000   551/10000

Then if starting from Standard grade, to get grade most likely to be is to look for 3rd row, which leads to that in 3 hours grade student most likely to get is of course Standard, and is around 0.45!

I have completed first task. Second I am unfamiliar, I don't know how to resolve it. What do we require in this process?

Answer 1

Hint: You should fine the stationary distribution (I guess) by solving the system $\pi P=\pi$ i.e. $$(\pi_1,\pi_2,\pi_3,\pi_4)\begin{pmatrix} 1/2 & 1/2 & 0 & 0 \\ 1/10 & 3/10 &3/5&0\\0&1/2&2/5&1/10\\1/10&0&4/5&1/10\end{pmatrix}=(\pi_1,\pi_2,\pi_3,\pi_4)$$ and $\pi_1+\pi_2+\pi_3+\pi_4=1$. Then multiply the stationary probabilities $\pi_i, i=1,2,3,4$ with 14, to find the percentage of students that are in the categories fail, acceptible, standard and genius.

If my calculations are correct then $\pi_1=\frac{8}{63},\pi_2=\frac{25}{63},\pi_3=\frac{27}{63},\pi_4=\frac{3}{63}$, and therefore in a class of $14$ the stationary states will be approximately $1.78$ fails, $6$ acceptible, $5.56$ standard and $0.67$ genius. If we have to round the numbers then it will be $2$ fails, $6$ acceptible, $5$ or $6$ standard and $1$ or $0$ geniuses respectively. (Please doublecheck the calculations because I am very prone to mistakes in matrix calculations).