There is a 6 sided die which is thrown repeatedly while summing the die values. If the sum exceeds 11, the sum is reset to 0, and throwing continues. If we get 11, we stop throwing. For example, the series of die throws 6, 6 (sum = 12 exceeds 11 --- reset sum to 0), 5, 6 (sum = 11, stop throwing) requires 4 die throws to get a sum of 11.
On average, how many die throws does it take to get a sum of 11?
Let $E(s)$ be the expected number of throws with current sum $s$.
$E(11) = 0$ by definition, and $E(s) = E(0)$ for $s>11$.
Then:
$$E(s) = 1 + \frac{1}{6}\sum_{k=1}^{6}E(s+k)$$
This yields a system of equations. Solve for $E(0)$.
For example:
$$E(7) = 1 + \frac{1}{6}(E(8) + E(9) + E(10) + E(11) + E(12) + E(13))$$
can be framed as
$$-6E(7) + E(8) + E(9) + E(10) + E(11) + E(12) + E(13) = -6$$
and then simplified to
$$-6E(7) + E(8) + E(9) + E(10) + E(11) + E(0) + E(0) = -6$$
or
$$2E(0) - 6E(7) + E(8) + E(9) + E(10) + E(11)= -6$$
which would correspond to the $E(7)$ row in the transition matrix.
The final matrix equation would be as follows:
$$\begin{bmatrix} -6 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & -6 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & -6 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & -6 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & -6 & 1 & 1 & 1 & 1 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & -6 & 1 & 1 & 1 & 1 & 1 & 1 \\ 1 & 0 & 0 & 0 & 0 & 0 & -6 & 1 & 1 & 1 & 1 & 1 \\ 2 & 0 & 0 & 0 & 0 & 0 & 0 & -6 & 1 & 1 & 1 & 1 \\ 3 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & -6 & 1 & 1 & 1 \\ 4 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & -6 & 1 & 1 \\ 5 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & -6 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} E(0) \\ E(1) \\ E(2) \\ E(3) \\ E(4) \\ E(5) \\ E(6) \\ E(7) \\ E(8) \\ E(9)\\ E(10) \\ E(11) \end{bmatrix} = \begin{bmatrix} -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ -6 \\ 0 \end{bmatrix}$$
You can use Gaussian elimination to solve the rest.