Find the transition function of the Markov chain (X_m)

370 Views Asked by At

I haven't taken a probability/statistics course in years and I'm trying to make my way through an Introduction to Stochastic Processes book. The question reads as follows:

Suppose we have two urns (a left urn and a right urn). The left urn contains $n$ black balls and the right urn contains $n$ red balls. Every time step you take one ball (chosen randomly) from each urn, swap the balls, and place them back in the urns. Let $X_m$ be the number of black balls in the left urn after $m$ time steps. Find the transition function of the Markov chain ($X_m$).

1

There are 1 best solutions below

0
On
  1. What are the possible values of $X_m$?
  2. For a certain value $x$ of $X_m$, what are the possible values of $X_{m+1}$? (Hint: you could pick two blacks, a red and a black, a black and a red, or two reds.)
  3. If $X_m$ is $x$, what is the probability that $X_{m+1}$ is $y$? (Hint: this depends on the probability of picking two blacks, red and black, black and red, or two reds, which is based on the number of blacks in the left urn.)
  4. Now you have computed the transition function so write it down.