Problem: A regular dice is thrown repeatedly. Which one of the following random variables with values in $\mathbb{N}\cup\{\infty\}\cup\{0\}$ is a Markov chain? For those who are, give the transition matrix:
1) The largest number up to the $n$-th throw
2) The number of fives up to the $n$-th throw
3) The time since the last 3 occured
4) The time until the next 1 occures
My attempt: I think it is implicitly assumed that the dice throws are independent. I am going to write $p_{ij}(n)$ for the transition matrix from state $i$ to state $j$.
1) This is a time-homogeneous Markov chain, since it only depends on the state directly before it (the state before "contains" the information of the past in some sense). The probabilities are given by $p_{ij} = \left\{ \begin{array}{l1} 0 & j \lt i \\ \dfrac{i}{6} & i = j \\ \dfrac{1}{6} & j \gt i \end{array} \right.$
2) This again is a time-homogeneous Markov chain, same reasoning as above, probabilities $p_{ij} = \left\{ \begin{array}{l1} \dfrac{5}{6} & i = j \\ \dfrac{1}{6} & j = i+1 \\ 0 & otherwise \end{array} \right.$
3) I'm assuming time here means the number of throws. This is also a time-homogeneous Markov chain, again same reasoning as for 1 and 2. probabilities $p_{ij} = \left\{ \begin{array}{l1} \dfrac{1}{6} & j = 0 \\ \dfrac{5}{6} & j = i+1 \\ 0 & otherwise \end{array} \right.$
4) I have no clue here. This doesn't even depend on the past, it depends on the future, so it has no chance to be a Markov chain.
I'm not entirely sure how I would go about formally reasoning that my clains are true. I think finding a transition matrix is enough to show that it is a Markov chain, since it directly gives us the stochastic kernel. How am I supposed to reason for 4)?