I have a markov process with states $S={0, 1, 2}$.
The goal is to calculate the expectation of $X_3$.
Given is:
\begin{equation} \begin{aligned} &P\left(X_0=0\right)=1 / 4 \\ &P\left(X_0=1\right)=1 / 4 \\ &P\left(X_0=2\right)=1 / 2 \end{aligned} \end{equation}
and:
\begin{equation} \mathbf{P}^3=\left(\begin{array}{ccc} 13 / 36 & 11 / 54 & 47 / 108 \\ 4 / 9 & 4 / 27 & 11 / 27 \\ 5 / 12 & 2 / 9 & 13 / 36 \end{array}\right) \end{equation}
From the definition of the expectation I construct the following equation.
\begin{equation} E\left(X_3\right)=0 \cdot P\left(X_3=0\right)+1 \cdot P\left(X_3=1\right)+2 \cdot P\left(X_3=2\right) \end{equation}
The problem is that I don't know how to calculate the terms: $P\left(X_3=i\right) $ with $i=1,2,3$.
I dont see how the given data connects to the asked quantity. What I know so far is that the $\mathbf{P}^3$ matrix shows the probability that $X$ goes from state $i$ to state $j$ in three runs.
Question
- How do the given probabilites of the values $X$ at $n=0$ connect to $P^3$?
- How do I find the unknown quantities $P\left(X_3=i\right) $
The transition matrix $P^3$ gives you the marginal probabilities $P(X_3 = i | X_0 = j)$. If you multiply these with the initial probabilities $P(X_0 = j)$, you can get the joint probabilities $P(X_3 = i, X_0 = j)$. Then, by summing those joint probabilities over values of $j$, you get the marginal probabilities $P(X_3 = i)$.
Or to put it another way, $P(X = x, Y = y) = P(X = x | Y = y)P(Y = y)$, and $P(X = x) = \sum_y P(X = x, Y = y)$.