Markov Chain and Polya's Urn Problem

721 Views Asked by At

An Urn contains a red and black ball.

At each time $n \ge 1$, a ball is taken randomly, its color noted, and both this ball and another ball of the same color are put back into the urn (ie. the number of balls in the urn increases by one ball each turn)

Continue similarly after n draws, the urn contains $n+2$ balls (+2 because we initially started with 2 ball)

$X_n = \{\text{the number of black balls after n draws} \}$

$X_n \in \{1,~2,~\cdots,~ n+1\}$

$X_0 =1$ (since at time zero we have two balls in urn and one of them is black)

$X_n$ is a (time-homogeneous) Markov Chain with transition probability:

  • urn increases by 1 black ball: $\Pr(X_{n+1}=k+1 ~\mid~X_n =k)$

  • urn increases by 1 red ball: $\Pr(X_{n+1}=k~\mid~X_n =k)$


Workbook says:

$$\Pr[X_{n+1} = k+1~\mid~ X_n = k] = \frac{k}{n+2}\tag{1}$$

$$\Pr[X_{n+1} = k ~\mid~ X_n = k ) = \frac{n+2+k}{n+2}\tag{2}$$

$$E[X_{n+1}~\mid~X_n] = X_n + \frac{X_n}{n+2}\tag{3}$$


How did they comes up with equation (2), and (3)?

Equation (1) makes sense to me since I would interpret it as (# black balls / total balls)

Equation (2) looks like an error to me... I would have thought that it would be (total balls - #black balls) / (total balls)

Equation (3).. well... not really sure how they came up with this one...

Polya's Urn