A die whose score cannot be as before (Markov chains)

1.9k Views Asked by At

A die is "fixed" so that each time it is rolled the score cannot be the same as the preceding score, all other scores having probability $1/5$. Given that the first score is 6, what is the probability that the $n$th score is 6 and what is it if the $n$th score is 1?

HINT: You can simplify things by selecting an appropriate state-space; do you really need a 6-state chain to answer the question?

Define $u_n(j) := P(X_n = j \mid X_1 = 6) = P_6 (X_n = j)$ and note that

$P_6(X_n = j) = \frac{1}{5} P_6(X_{n-1} \neq 6) = \frac{1}{5} (1 - P_6(X_{n-1} = 6))$

So $u_n = \frac{1}{5} (1 - u_{n-1})$ and solving the difference equation for the cases $j=6,1$ we have

$u_n(6) = \frac{1}{6} (1+5(-\frac{1}{5})^{n-1})$

$u_n(1) = \frac{1}{6} (1-(-\frac{1}{5})^{n-1})$

However, I feel that by using difference equations to obtain the solutions I am missing the point of the question and the hint. I tried selecting several state-spaces but couldn't get anywhere. Can anyone shed some light on this?

1

There are 1 best solutions below

0
On BEST ANSWER

Hint: Define the random variable with $Y_n \in \{0,1\}$ (indicator variable with $2$ states) as follows $$Y_n=\begin{cases}1,& X_n=6\\0,&X_n\neq6\end{cases}$$ with transition matrix $$\mathbf P_{(1)}=\begin{array}{r|cc|r}&0&1&\\\hline0&\frac{4}{5}&\frac{1}{5}\\1&1&0 \end{array}$$ Initially $Y_0=1$. You want to find the probability $$P(Y_n=1|Y_0=1)$$ which can be done by calculating the matrix $\mathbf P_{(n)}=P^n$.


Indeed using diagonalization method (or preferably online calculators...) you find that $$\mathbf P_{(1)}=\begin{array}{r|cc|r}&0&1&\\\hline0&\frac{5}{6}+\frac{1}{6}\cdot\left(-\frac{1}{5}\right)^n&\frac{1}{6}-\frac{1}{6}\cdot\left(-\frac{1}{5}\right)^n\\1&\frac{5}{6}-\frac{5}{6}\cdot\left(-\frac{1}{5}\right)^n&\frac{1}{6}+\frac{5}{6}\cdot\left(-\frac{1}{5}\right)^n \end{array}$$

which gives you that $$p_{1,1}^{(n)}=\frac{1}{6}+\frac{5}{6}\cdot\left(-\frac{1}{5}\right)^n$$ and $$p_{1,0}^{(n)}=\frac{5}{6}-\frac{5}{6}\cdot\left(-\frac{1}{5}\right)^n$$ confirming your results (note that you have to multiply $p_{1,0}^{(n)}$ with $1/5$ to get the probability that $X_n=1$, since $Y_n=0$ implies only that $X_n\neq0$ and then there is equal probability $1/5$ for each result in $\{1,2,3,4,5\}$).