A friend of mine was telling me about one of the problems, which he described thus:
As you can see, the answer to the toy problem presented here is reportedly 13. However, I don't understand how they arrive at that number.
As I understand it, this diagram is a Markov chain and the problem can be solved by premultiplying the initial-state vector
$$ x_0 = [1,0,0]^T $$
by the following (left) stochastic matrix
$$ T = \begin{bmatrix} 0.5 & 0.75 & 0 \\ 0.5 & 0 & 0 \\ 0 & 0.25 & 1 \\ \end{bmatrix} $$
(where $T_{ij}$ corresponds to the probability of a transition to state $i$ while in state $j$) as many times ($k$) as it takes for the last entry in the state vector to be $\geq 0.5$. The answer is then $k+1$, because you need to add $1$ for the very first turn (the one that initially takes you to Square 1).
According to my reasoning, the answer should be 10, not 13. How did they arrive at 13?
Multiplying an initial vector by the k-th power of the transition matrix you will only get the distribution of your state at step k. You need to learn/return to the basics of probability theory and Markov chains. I can recommend Grimmett and Stirzaker's book Probability and Random Processes.