Calculate probability of two different outcomes where history is governed by markov chain

65 Views Asked by At

Let the state space, $s_t$, be $\{0,1\}$ and be governed by a Markov chain with probability $\pi(s_0=1) =1$ for the initial state and time-varying transition probabilities

$\pi_1(s_1=1|s_0=1)=1$,

$\pi_2(s_2=1|s_1=1)=1 =\pi_2(s_2=0|s_1=1)=.5$ and

$\pi_t(s_t = 1|s_{t-1}=1) = \pi_t(s_t = 0|s_{t-1}=0)=1 \text{ for }t>2$.

This specification implies that $$ \tag{1} \pi_t(1,1,\dots,1,1,1) = .5 \text{ and } \pi_t(0,0,\dots,0,1,1) = .5 $$ for all $t>2$

How is $(1)$ calculated?

For $\pi_t(1,1,\dots,1,1,1)$ my guess is that $\pi_t(1) =1, \pi_t(1,1)=1, \pi_t(1,1,\dots) = .5$ because the probability of the third event is half, and everything afterwards has probability $1$?

I have no idea how we can calculate, or even discuss, $\pi_t(0,0,\dots,0,1,1)$ since the initial state is $1$ with probability one, and the probability of switching from $0$ to $1$ at the end seems to be $0$ to me... so that has me stumped.

Thanks

Also, perhaps worth noting is that $\pi_t(s^t)$, where $s^t$ is a sequence, is the probability of observing that particular sequence of events. The probability of observing $s^t$ conditional on the realization of $s^\tau$ is denoted as $\pi_t(s^t|s^\tau), t>\tau$

1

There are 1 best solutions below

0
On BEST ANSWER

With a markov chain, $$ p_{s^t}=p_{s_t|s_{t-1}}\cdot p_{s_{t-1}|s_{t-2}}\cdot \dots\cdotp_{s_{1}|s_{0}}p_{s_0} $$ This follows from the general chain rule for chain PMF's. $$\tag{2} p_{X_1,X_2,\dots ,X_n} = p_{X_n|X_1,X_2,\dots , X_{n-1}}p_{X_{n-1}|X_1,X_2,\dots , X_{n-2}}\dots p_{X_2|X_1}p_{X_1} $$ However, for a markov chain as in my question, since $p_{X_n|X_1,X_2,\dots , X_{n-1}} = p_{X_n|X_{n-1}}$, (2) reduces to $$ \tag{3} p_{X_1,X_2,\dots ,X_n} = p_{X_n|X_{n-1}}p_{X_{n-1}|X_{n-2}}\dots p_{X_2|X_1}p_{X_1} $$

In my question, $\pi_t$ is the probability of something occurring at time $t$. However, since there is only one uncertain event, $\pi_2$, $(3)$ will always be $.5$, which I now realize this is what $(1)$ in my question is probably saying.

Lastly, $\pi_t(0,0,\dots,0,1,1)=\pi(s_0=1,s_1=1,s_2=0,\cdots,s_{t-1}=0,s_t=0)$, which we can use $(3)$ to calculate.