How do you express the probability of outcomes for the next state in Markov chain (given some state)?

Question

How do you express the probability of outcomes for the next state in Markov chain (given some state)?

This is an expected value, i.e. suppose 5 states, then if one starts at state 4, then the "outcomes" from this are:

$$\sum_{k=1}^5 x_k \mathbb{P}(f_1=k|f_0=4)$$

However, I have troubles, regarding, how is this expressed using the $\mathbb{E}(...)$ syntax?

Answer 1

What you wrote down is how the conditional expectation is defined.

$$\mathbb{E}[X|Y=y] = \sum^n_{k=1} x_k\mathbb{P}(X = x_k | Y = y)$$

Answer 2

You are computing the expectation of some function of the Markov chain. Namely, $X_t = \sum_{k=1}^5 x_k 1\{f_t = k\}$. In this case, your expression is just $\mathbb{E}[X_1 \mid f_0 = 4]$.

The way to obtain this is to write $\mathbb{P}(f_1=k\mid f_0=4)$ as $\mathbb{E}[1\{f_1=k\} \mid f_0=4]$ and use linearity of expectation.