About the expected transitions in Markov Chain

234 Views Asked by At

The problem is here: enter image description here

The given answer is here:

K = $2+ X_1 + X_2$, where $X_1$ and $X_2$ are independent exponential random variables with parameters $2/3$ and $3/5$. $$ E[K] = 2=2+1/p_1 +1/p_2 = 31/6 $$ $$var(K)= 1 −\frac{1-p_1}{p_1^2}- \frac{1-p_2}{p_2^2} = 67/36 $$

I am confused. 1. The Markov chain is discrete. The exp. distribution is applied in continuous case. I think geo. distribution is better. 2. Why is exp. distribution valid?

Thanks for your help!

1

There are 1 best solutions below

1
On BEST ANSWER

You're certain to take exactly one step each from $1$ to $2$ and from $3$ to $4$; that's the constant $2$. At $2$, you have a Bernoulli experiment with success probability $\frac23$ that you perform until it succeeds, and at $4$ you have a similar Bernoulli experiment with success probability $\frac35$. The expected time until a Bernoulli experiment succeeds is the reciprocal of the success probability.