Two players must choose a number simultaneously between 0 and 1. If x+y≤1 they both get utility equal to the numbers that they have chosen. Otherwise both get 0 utility.
I think that the best response functions are:
Player 1: x=1-y
Player 2: y=1-x
If I substitute I get that x=x and y=y. My guess is that both players choose 1 but I don't know how to prove it.
How do I find the Nash equilibria and expected payoff?
It is a bit more complicated, best responses are not unique if the other player plays $1$. In that case, everything gives a payoff of $0$.But for $x\in(0,1)$ you are fine. So let $x\in (0,1)$ and look at the pair of actions $(x,1-x)$. Is $x$ a best response to $1-x$? Is $1-x$ a best response to $x$?Then you know whether $(x,1-x)$ is a Nash equilibrium. Are there other best responses?
Coming to the boundary cases, we know that for $x\in (0,1)$, the best response is $1-x\in(0,1)$, so there is no Nash equilibrium in which only one player plays $1$. But $(1,1)$ is indeed a Nash equilibrium; no player can avoid getting a payoff of $0$ by unilaterally taking a different action.
Characterizing Nash equilibria in general mixed strategies is a different, much harder, problem.