Discrete-Time Markov Chain example

101 Views Asked by At

A machine produces a great, good or defective set of parts. If a defective set of parts is produced, then the process is stopped for the time it takes to create one set of parts and the machine is readjusted during this time. When the machine is back up and running the next set will be good and great with equal probability. If the machine produced a great set of parts, then the machine is equally likely to produce either a good set or a great set next. If the machine produces a good set of parts, then it will produce a good set 70% and a bad set 30% of the time.

What proportion of time is the machine getting reset?

1

There are 1 best solutions below

0
On BEST ANSWER

What you are looking for is the stationary distribution of the markov chain with the transition Matrix $$M=\left(\begin{array}{cccc} \frac{1}{2} & \frac{1}{2} & 0 & 0\\ 0 & \frac{7}{10} & \frac{3}{10} & 0\\ 0 & 0 & 0 & 1\\ \frac{1}{2} & \frac{1}{2} & 0 & 0 \end{array}\right)$$ where the states correspond to the states of the machine $\{great, good, defect, reset\}$. to compute the stationary distribution you have to solve $\pi M=\pi$ for pi and then normalize $\pi$ with respect to the summation norm. Matlab gives me for eigenvalue 1 the eigenvector $\pi\simeq (0.2662,0.8874,0.2662,0.2662)$ the summation norm of which is given by $\Vert \pi\Vert_{L^1}\simeq 1.6859$ hence the stationary probability of observing a $reset$ in the production is $$\mathbb{P}_\pi(reset)=\frac{\pi^{reset}}{\Vert \pi\Vert_{L^1}}\simeq 0,1578.$$