To show that $\{Y_n:n\ge1\}$ is a Markov Chain.

86 Views Asked by At

$\{X_n : n\ge1\}$ are I.I.D Bernoulli random variables with probability of success being $1-p$

i.e $P(X_i = 1) = 1-p$

Define: $Y_n = \min\{M, X_1+X_2+\ldots+X_n\}$

Is $\{Y_n:n\ge1\}$ a Markov Chain?

$\textbf{My Intuition}$: $Y_n$ can be either $M$ or a number between $0$ and $n$ depending on the values of $X_i$'s. If $Y_{n-1} = M$ then $Y_n = M$ only no matter what the value of $X_n$ is. If $Y_n = X_1+\ldots+X_{n-1}$ then it depends on the value of $X_n$, what the value of $Y_n$ would be. Therefore $Y_n$ would depend only on $Y_{n-1}$ and none of the other past values and hence it is a Markov Chain. But how to prove this, the definition says: $\mathbb{P}[Y_n=y_n\mid Y_{n-1}=y_{n-1},...,Y_1=y_1]=\mathbb{P}[Y_n=y_n\mid Y_{n-1}=y_{n-1}]$ How to find such probability? Any help would be appreciated.

$\textbf{Edited}$: To prove Markov chain , I have to express $Y_{n+1}$ as a function of $Y_n$. $Y_{n+1} = \cases{ M & if $Y_n =M$ \\ \min\{M, X_1+X_2+\ldots+X_{n+1}\} & if $Y_n < M$ }$

But is this correct and enough to prove that $Y_n$'s follow markov chain? Also I want to know of what order transition probability matrix would be?

1

There are 1 best solutions below

2
On

If $y_i=M$ for some $i<n$ then, conditioned on $Y_i=y_i$, we get $X_1+X_2+...,X_i \ge M$ which implies $X_1+X_2+...+X_n \ge M$ and $Y_n=M$. So $\mathbb{P}[Y_n=y_n\mid Y_{n-1}=y_{n-1},...,Y_1=y_1]=1$ or $0$ depndning on whether $y_n=M$ or not.

If $y_i \neq M$ for all $i \leq n$ then $\mathbb{P}[Y_n=y_n\mid Y_{n-1}=y_{n-1},...,Y_1=y_1]=\mathbb{P}[Z_n=y_n\mid Z_{n-1}=y_{n-1},...,Z_1=y_1]$ where $Z_i=X_1+X_2+...+X_i$. You know how to write this down, right?

If $y_i \neq M$ for all $i < n$ but $y_n=M$ then we have to compute $\mathbb{P}[Z_n \ge M\mid Z_{n-1}=y_{n-1},...,Z_1=y_1]$. I will leave the rest to you.