What does the notation $(p^{(i)}(x_k))_{k=1,...,M}$ mean?
This is a marginal distribution. But I have troubles understanding whether this is a vector or a product or something? I think the notation refers to sequence notation as in $(x_k)_{k=1,...,M}$.
$$p^{(i)}(x_k)=\mathbb{P}(f_i=x_k)$$
So does the notation specify:
$$(\mathbb{P}(f_i=x_1),...,\mathbb{P}(f_i=x_M))$$
or something else?
For all $i \geq 0$, $p^{(i)}$ is a probability distribution on the states $\{x_1,\dots,x_M\}$, where $p^{(i)}(x_k)=P(f_i=x_k)$. That is, $p^{(i)}$ describes the distribution of the random variable $f_i$. Writing $(p^{(i)}(x_k))_{k=1,\dots,M}$ is indeed shorthand for the entire list of probabilities $(p^{(i)}(x_1),\dots,p^{(i)}(x_M))=(P(f_i=x_1),\dots,P(f_i=x_M))$.