In the following game I found one pure nash equilibrium: $(R, r)$:
$\begin{array}{r|ccc} A\backslash B & l & m & r\\ \hline L & (-10, 4) & (10, 0) & (-1, -1)\\ M & (0, 10) & (-1, -1) & (-1, 1)\\ R & (4, -10) & (-1, -1) & (2, 2) \end{array} $
Now I'm trying to find all the mixed nash equilibrium:
Let $l$ probability be $p_1$, $m$ probability be $p_2$, and $r$ probability be $1-p_1-p_2$.
$EU(L) = -10p_1 + 10p_2 -1(1-p_1-p_2)$
$EU(M) = 0p_1 - p_2 -1(1-p_1-p_2)$
$EU(R) = 4p_1 - p_2 + 2(1-p_1-p_2)$
And the expected utility of all options should be equals.
But by solving $EU(L)=EU(M)=EU(R)$ I'm getting $p_2 = \dfrac{30}{19}$ which is greater than $1$ and not making any sense.
What am I doing wrong?
Thanks!
This game has a unique pure strategy Nash equilibrium that you already found, $(R,r)$.
The reason you cannot find a probability distribution (mixed strategy) for the column player that equalizes the expected payoffs of all three rows is because this is not possible.
One approach to finding all extreme Nash equilibria of a bimatrix game is to try all pairs of supports. For a given support of player 1, say $S_1$, which is a set of pure strategies, and one for player 2, $S_2$, you try to find a mixed strategy $y$ of player 2 using only pure strategies in $S_2$ that makes the pure strategies in $S_1$ have equal expected payoff $p$. If you can do this then you also need to check that $p$ is greater than the expected payoff against $y$ of the pure strategies not in $S_1$.
You have found that in this 3x3 game, when $S_1 = \{L,M,R\}$ there is no mixed strategy of player 2 (for any support) that equalizes the expected payoffs of the pure strategies in $S_1$, and thus there is no Nash equilibrium that uses the support $S_1$.
In this game, the only pair of supports that yields a Nash equilibrium is the pair of singletons $S_1 = \{R\}$, $S_2 = \{r\}$.
To confirm this, you can use the following online game solvers:
http://banach.lse.ac.uk/
or
http://www.gametheoryexplorer.org/
Here is sample output from the first website.
3 x 3 Payoff matrix A: -10 10 -1 0 -1 -1 4 -1 2 3 x 3 Payoff matrix B: 4 0 -1 10 -1 1 -10 -1 2 EE = Extreme Equilibrium, EP = Expected Payoff Decimal Output EE 1 P1: (1) 0.0 0.0 1.0 EP= 2.0 P2: (1) 0.0 0.0 1.0 EP= 2.0 Rational Output EE 1 P1: (1) 0 0 1 EP= 2 P2: (1) 0 0 1 EP= 2 Connected component 1: {1} x {1}