Player 1 rolls a 10 sided fair die. Player 2 rolls a 20 sided fair die. Player 2 is allowed to reroll a second time, but is not allowed to look at player 1's roll before deciding whether they want to reroll or not. If player 2 rerolls, they forfeit the number on their first toss and the game is decided based on player 2's second toss and player 1's toss. Player 1 wins if player 1's toss is greater than player 2's. Player 2 wins if player 2's toss is greater or equal to player 1's toss. The loser pays the value of the other player's toss. At what threshold should player 2 decide to reroll or not?
I think the idea is to minimize the expected value of player 1. I think minimizing the probability of player 1 winning wouldn't give us an equivalent answer. So I'm going to proceed based on what we're going to minimize the expected value of player 1. Apparently the threshhold is $7$ and there's some fairly "straightforward" way to arrive at this answer, but I don't see what that could be without writing out a bunch of conditional expectations.
Define $A, B$ to be the result of player 1 and 2's toss, respectively. Denote $N$ to be player 1's profit.
We have
$$ E[N] = E[N | A > B]P(A > B) + E[N | A \leq B] P(A \leq B) $$
The first term is positive, and the second term is negative. I don't see an easier way to approach this problem besides further conditioning each of those conditional expectations on whether $B$ rerolls or not. Let $B_1$ denote the result of $B$'s first toss. $B$ will reroll if $B_1 < b$ and not reroll otherwise, so we have:
$$ E[N | A > B]P(A > B) = E[N | A > B, B_1 < b]P(B_1 < b | A > B) + E[N | A > B, B_1 \geq b]P(B_1 \geq b | A > B) \\ E[N | A \leq B]P(A \leq B) = E[N | A \leq B, B_1 < b]P(B_1 < b | A \leq B) + E[N | A \leq B, B_1 \geq b]P(B_1 \geq b | A \leq B) $$
So, we need to solve
$$ \arg\min_b E[N] = E[N | A > B, B_1 < b]P(B_1 < b | A > B) + E[N | A > B, B_1 \geq b]P(B_1 \geq b | A > B) + E[N | A \leq B, B_1 < b]P(B_1 < b | A \leq B) + E[N | A \leq B, B_1 \geq b]P(B_1 \geq b | A \leq B) $$
Is this the right approach? Is there an easier approach to this problem?
Here's what I obtained for the conditional expectations (sorry kind of scratch work so far):
We have 4 conditional expectations to solve. Let's look at $E[N | A > B, B_1 < b]$ first. Here, player 2 will reroll. In this case, how many situations are there such that $A > B$? Well, there are $1 + 2 + 3 + \ldots + 9 = 45$ situations. For this expectation, we don't need to worry about what the threshold $b$ is.
\begin{align} E[N | A > B, B_1 < b] = \frac{1}{45}\sum_{i=2}^10 i * (i - 1) = 330/45 = 7\frac{1}{3} \end{align}
For the 3rd conditional expectation, we need to consider the situations where we reroll and we have $A \leq B$. There are $20 + 19 + 18 + \ldots + 11 = 155$.
\begin{align} E[N | A \leq B, B_1 < b] = -\frac{1}{155}\left[\left(\sum_{i=1}^{10} i * i \right) + \sum_{i=11}^{20} 10i \right] \\ = -12.4838 \end{align}
Now the second and fourth conditional expectations require us to consider the threshhold. We know the threshhold is not going to be above for sure. Let's consider the second conditional expectation first. Consider $A > B, B_1 \geq b$. There are $20 - b + 1$ possibilities for $B_1$, i.e., $b, b+1, b+2, \ldots, 20$. However, to have $A > B$, we must only have $B_1 \leq 10$. So the number of events with $A > B, B_1 \leq b$ should be $10 - b, 10 - b - 1, \ldots, 1, 0 = \frac{(10-b)(10-b+1)}{2}$
\begin{align} E[N | A > B, B_1 \geq b] = \frac{2}{(10 - b)(10 - b + 1)} \sum_{i=b + 1}^{10} i * (i - b) \end{align}
Now for the fourth expectation conditional expectation, we need to consider $A \leq B, B_1 \geq b$. There are $b + b+1 + b+2 + \ldots + 10 + 10 * 10 = 100 + \frac{10*11}{2} - \frac{b(b-1)}{2} = 155 - 0.5b^2 + 0.5b$ such events. So we have
\begin{align} E[N | A \leq B, B_1 \geq b] = - \frac{1}{155 - 0.5b^2 + 0.5b} \left[ \left(\sum_{i=b}^{10} i^2 \right) + \sum_{i=11}^{20} 10i \right] \end{align}
Now we have to compute the conditional probabilities. \begin{align} P(B_1 < b | A > B) = \frac{P(A > B | B_1 < b)P(B_1 < b)}{P(A > B)} \\ = \frac{\frac{45}{200}\frac{b - 1}{20}}{\frac{45}{200}\frac{b - 1}{20} + \frac{(10-b)(10-b+1)/2}{(20-b+1)(10)}\frac{20 - b + 1}{20}} \end{align}
Now we can get the second conditional probability by taking the complement
\begin{align} P(B_1 \geq b | A > B) = 1 - \frac{\frac{45}{200}\frac{b - 1}{20}}{\frac{45}{200}\frac{b - 1}{20} + \frac{(10-b)(10-b+1)/2}{(20-b+1)(10)}\frac{20 - b + 1}{20}} \end{align}
Now for the third conditional probability
\begin{align} P(B_1 < b | A \leq B) = \frac{P(A \leq B | B_1 < b)P(B_1 < b)}{P(A \leq B)} \\ = \frac{ \frac{155}{200} \frac{b - 1}{20}}{ \frac{155}{200} \frac{b - 1}{20} + \left(1 - \frac{(10-b)(10-b+1)/2}{(20-b+1)(10)} \right) \frac{20-b+1}{20} } \end{align}
And the fourth
\begin{align} P(B_1 \geq b | A \leq B) = 1 - \frac{ \frac{155}{200} \frac{b - 1}{20}}{ \frac{155}{200} \frac{b - 1}{20} + \left(1 - \frac{(10-b)(10-b+1)/2}{(20-b+1)(10)} \right) \frac{20-b+1}{20} } \end{align}
Conclusion: I use monte carlo and obtained $b = 10$ as a threshhold, so if $B$ rolls less than $10$ then they should reroll. Obviously $b$ can't be more than 10.
I am supposed to discuss your work, before presenting my answer. Unfortunately, while I can solve the problem, my formal knowledge of Probability is inadequate to examine your work.
On the re-roll, Player 2's expectation is as follows:
$(1/2)$ the time, he will roll $> 10.$ If he does so, his average gain is $(15.5)$.
$(1/20)$ of the time, his roll will exactly match player 1. When this happens, player 2's average gain will be $(5.5).$
The other $(9/20)$ of the time, both he and player 1 will roll $10$ or less, and their rolls won't match. This case may be ignored, since by symmetry, it must be a break even situation.
Therefore, player 2's expectation is
$[(1/2) \times (15.5)] + [(1/20) \times (5.5)] = \frac{160.5}{20}.$
If player 2 rolls a $9$, and does not re-roll, his expectation is
$[(.9) \times 9] - [(1/10) \times (10)] = 7.1$
If player 2 rolls a $10$, and does not re-roll, his expectation is
$[(1.0) \times 10] = 10$
Player 2 should re-roll on 9 or less.
Addendum
Responding to comment/question.
Player 1's expectation, assuming that player 2 re-rolls on 9 or less is simply $(-1) \times $ player 2's expectation.
Player 2's expectation is :
$(0.45) \times \frac{160.5}{20}.$
This represents player 2 re-rolling on a $9$ or less.
$(0.55) \times 15.$
This represents player 2 rolling 10 or higher.
Therefore, player 2's expectation, based on the reroll on 9 or less strategy is:
$$\left[(0.45) \times \frac{160.5}{20}\right] + \left[(0.55) \times 15\right].$$