Writing down the transition matrix of a discrete Markov chain

345 Views Asked by At

Please consider the following scenario:

  • One person is walking along a discrete circle induced by $\mathbb{Z}/n\mathbb{Z}$
  • In each round we roll a dice with $w\in\left\{2,\ldots n\right\}$ sides
  • If the dice yields $\delta\in\left\{1,\ldots, w\right\}$ and the person is currently placed at position $\overline{i}$ he walks along the circle up to position $\overline{i+\delta}$

What I want to do, is modeling this scenario as a Markov chain. I get stuck when trying to write down the corresponding transition matrix $$P(t)=\left(p_{ij}(t)\right)_{0\le i,j<n}\;\;\;\text{with}\;\;\;p_{ij}(t):=P\left(X_{t+1}=j\mid X_t=i\right)$$ where $X_t$ is the position of the person at time $t$.

I wondered whether the events $X_{t+1}=j$ and $X_t=i$ are statistically independent or not. However, the main question is: How do we write down $P(t)$?

1

There are 1 best solutions below

0
On BEST ANSWER

Let's look at a specific case; e.g., suppose $n = 5$ and $w = 3$. Then $$\mathcal P = \begin{bmatrix}0 & 1/3 & 1/3 & 1/3 & 0 \\ 0 & 0 & 1/3 & 1/3 & 1/3 \\ 1/3 & 0 & 0 & 1/3 & 1/3 \\ 1/3 & 1/3 & 0 & 0 & 1/3 \\ 1/3 & 1/3 & 1/3 & 0 & 0 \end{bmatrix}.$$ As you can see, it is never possible to remain in state $i$ given that one is in currently in state $i$, because the value on the die is one of $\{1, 2, 3\}$ with equal probability.

It should be easy, from this example, to extrapolate the form of $\mathcal P$ for general integers satisfying $2 \le w \le n$.