Let $(Y_i)_{i\geq0}$ be i.i.d. random variables, following the uniform distribution in state space S = $\{1,...,6\}$. The goal is to show that the following sequence of random variables is a Markov Chain and compute both the initial distribution and the transition martix.
- the largest number $L_n$ shown so far.
First, we know the present, which is: $L_n = max\{Y_1,...,Y_n\}$. Then the future is $L_{n+1}=max\{L_n,Y_{n+1}\}$, which is independent of the past $\{L_0,...,L_{n-1}\}$, hence this sequence is indeed a Markov Chain.
- Initial distribution: $\mathbb{P}(L_0=x_0)=\mathbb{P}(Y_0=x_0)=1/6$ for $x_0\in S$ fixed.
- Transition Matrix: Fix $x,y\in S$, Q(x,y)=$\mathbb{P}(L_{n+1}=y|L_n=x)= \begin{cases} 0\quad if \quad y<x,\\ \frac{x}{6}\quad if \quad y=x \\ \frac{1}{6}\quad if \quad y>x \end{cases} $
Here are my questions:
- To get $\frac{1}{6}$ we need to compute: $\mathbb{P}(L_{n+1}=y|L_n=x)$. How do we compute this and get 1/6?
- To get $\frac{x}{6}$ we have to compute: $\mathbb{P}(L_{n+1}=x|L_n=x)=\mathbb{P}(Y_{n+1}\leq x)$. How do we come to $\mathbb{P}(Y_{n+1}\leq x)$ and then how do we get to $\frac{x}{6}$?
Thank you in advance.