I do not understand how to calculate the expected utility of $3$ or more players game.
For a $2$ player game, it is easy. Suppose I have two action $\{A, B\}$ and my opponent has two action $\{C, D\}$.
If I want to calculate my expected payoff if I play $A$ and my opponent mixes the play like this: with probability $p$, he plays $C$ and with probability $1-p$, he plays $D$, then I will get $\hat u=p\cdot u(A,C)+(1-p)\cdot u(A, D)$.
How to calculate the expected payoff for $3$ players game? Let say player $1$ has strategy $\{A, B, C\}$, player $2$ mix $\{C, D, E\}$ with probabilities $p$, $q$, and $1-p-q$ and player $3$ mix $\{F, G, H\}$ with probabilities $r$, $s$, and $1-r-s$.
Let's just say all the strategies are $s_{i}\in\{A,B,C\}\equiv\mathcal{S}_{i}$ for players $i=1,2,3$
Denote player $i's$ mixing probabilities over $\{A,B,C\}$ as $p_{i}^{A},p_{i}^{B},p_{i}^{C}$
Expected utility of player 1 playing strategy A is:
$E(U(A))=p_{2}^{A}p_{3}^{A}U(A,A,A)+p_{2}^{A}p_{3}^{B}U(A,A,B)+p_{2}^{A}p_{3}^{C}U(A,A,C)+etc...$