Assume a die is rolled repeatedly. Find the markov matrix $P$ for the random variable of the time until the next $6$.

Question

Assume a die is rolled repeatedly. Find the markov/transition matrix $P$ for the random variable $X_r$ = the time until the next six at time $r$.

My solution was:

For $i,j \geq 0$, $P$ is given by $$ p_{ij} = \begin{cases} 1 & j = 1 - i, i \geq 1 \\ (1/6)(5/6)^{j} & i = 0. \\ 0, & \text{otherwise}. \end{cases} $$

But I am told the correct solution is in fact:

For $i,j \geq 0$, $P$ is given by $$ p_{ij} = \begin{cases} 1 & j = 1 - i, i \geq 1 \\ (1/6)(5/6)^{j-1} & i = 0. \\ 0, & \text{otherwise}. \end{cases} $$

Notice the difference in the power $(5/6)$ is raised to. What is the correct answer? I have trouble believing it's the latter, since it's always possible to roll two sixes in a row, hence the probability $P(X_{n+1} = 0 | X_n = 0)$ should always be strictly greater than zero and in fact equal to $1/6$.

Can someone point out the flaw in my reasoning?

Answer 1

For the transition matrix, we require that P be row-stochastic, so for all i,

$\sum\limits_{j}p_{ij} = 1$.

In particular, for $i=0$, since $\sum\limits_{j=0}^{\infty}{(\frac{5}{6})^j} = \dfrac{1}{1 - \frac{5}{6}} = 6$

only the first definition (with the exponent of $j$ not $j-1$) makes sense. The exponent $j-1$ would only make sense if the middle condition was amended from $i=0$ to $i = 0,j \ge 1$.