You are playing a game where you a paid the product of 2 dice
a) How much would you pay to have the opportunity to reroll any 1 of the dice?
b) How much would you pay to roll 3 dice and discard the lowest?
I know the EV of product of 2 dice is 12.25
I calculated the Probability of the Product of 2 dice to be <= 12 as $ \frac {23} {36} $
Similarly the probability of the Product of 2 dice to be > 12 as $ \frac {13} {36} $
Now for Part a) New E.V. = $ \frac {23} {36} $ * (Increase in EV in this case) + $ \frac {13} {36} $ * (0 Increase)
However, I'm confused how to approach the Increase in EV in the case where we choose to reroll 1 of the dice with Probability of 23/36.
One approach I can think of for the above case is that we would only reroll a 1,2 or 3 and never a 4,5,6 as (4 * 4 = 16), (5 * 5 = 25), (6*6 = 36) is > than 12.25.
Then - $$ \frac {1} {3} * (3.5-1) + \frac {1} {3} * (3.5 -2) + \frac {1} {3} * (3.5-3) = 1.5 $$
Now, I'm not sure if it should be $ \frac {1} {3} or \frac {1}{6} $ for the probabilities above considering that it is conditioned on rerolling only if we get a 1,2 or 3
Now the 2nd die E[x] = 3.5 so we can say $$ 1.5 * 3.5 = 5.25 $$ which would be the increase in EV in the case where we choose to reroll 1 of the die
Hence increase in EV = $$ \frac {23} {36} * 5.25 = \frac {161} {48} $$
Not sure if this approach is correct
Part b) I'm not sure how to approach it
(a) Given the opportunity to reroll a singular dice, we note that this only makes sense to do if the number it is currently showing is $3$ or less, as the expected value of the reroll is $3.5$. Hence, the new expected value of a roll of $(m, n)$ is $3.5 \max(m, n)$ if any of $m$ or $n$ are $\leq 3$, and otherwise it remains $mn$. Splitting into $4$ cases, we have
Case 1: $m, n \geq 4$
In this case, we choose not to reroll, so the total sum of products is $(4+5+6)(4+5+6) = 15^2$.
Case 2: $m \geq 4$, $n \leq 3$
In this case, we choose to reroll the $n$-die, so the total average sum of products is $(4+5+6)(3\times 3.5) = 3 \times 3.5 \times 15$.
Case 3: $m \leq 3$, $n \geq 4$
This case is virtually the same as previous, and we get the same total.
Case 4: $m, n \leq 3$
In this case, we choose to reroll the smaller die, so the total average sum of products is $5 \times 3.5 \times 3 + 3 \times 3.5 \times 2 + 1 \times 3.5 \times 1$.
this gives an overall expected value of $$\frac{15^2}{36} + 2 \cdot \frac{3 \times 3.5 \times 15}{36} + \frac{5 \times 3.5 \times 3 + 3 \times 3.5 \times 2 + 1 \times 3.5 \times 1}{36} = \boxed{\frac{617}{36}}$$ Hence, you should be willing to pay up to $\frac{617}{36}$ to play this game (or $\frac{44}{9}$ more than the game without any reroll).
(b) We break this up into cases depending on what the smallest number is:
Case 1: Smallest number is $1$.
There are $91$ possible ways of this occurring, with total sum of product being $1 \times 1^2 + 3 \times 1 \times 20 + 3 \times 20^2 = 1261$.
Case 2: Smallest number is $2$.
There are $61$ possible ways of this occurring, with total sum of product being $1 \times 2^2 + 3 \times 2 \times 18 + 3 \times 18^2 = 1084$.
Case 3: Smallest number is $3$.
There are $37$ possible ways of this occurring, with total sum of product being $1 \times 3^2 + 3 \times 3 \times 15 + 3 \times 15^2 = 819$.
Case 4: Smallest number is $4$.
There are $19$ possible ways of this occurring, with total sum of product being $1 \times 4^2 + 3 \times 4 \times 11 + 3 \times 11^2 = 511$.
Case 5: Smallest number is $5$.
There are $7$ possible ways of this occurring, with total sum of product being $1 \times 5^2 + 3 \times 5 \times 6 + 3 \times 6^2 = 223$.
Case 6: Smallest number is $6$.
There is $1$ possible ways of this occurring, with total sum of product being $1 \times 6^2 = 36$.
Hence, the expected value of the game is $\boxed{\frac{1967}{108}}$, so this is the maximum you should consider paying.