Consider the following game matrix: $\begin{aligned}[] \begin{array}{|c|c|c|} \hline & A & B \\ \hline C & 5,0 & 1, 2 \\ \hline D & 1,2 & 7,4\\ \hline \end{array} \end{aligned}$
The pay-offs are units of money. The higher the pay-off, the more money you get. The player who ends up with more money wins.
What's the best response against $D$?
If I go with $A$, then $u_2(D,A) = 2 > 1 = u_1(D,A)$ so the second player wins and if I go with $B$, then $u_2(D,B) = 4 > 2 = u_2 (D,A)$ which probably makes it the "Best Response" but the second player loses. So the question is, how is the second one the "BR" where the player is losing?
You seem to misunderstand the notion of best response. We say strategy $s^*$ is a best response for player $i$ to $s_{-i}$ when for all $s$ in player $i$'s strategy set,
$$u_i(s^*,s_{-i})\geq u_i(s,s_{-i}).$$
That is, $s^*$ gives at least as high a payoff as any other strategy for player $i$, given all others play according to $s_{-i}$.
In your example, $B$ is the best response to $D$ as you point out since
$$u_2(D,B)=4\geq 2=u_2(D,A).$$
It makes no sense whatsoever to compare payoffs between players. Remember that payoffs are simply utility numbers and can be scaled. For instance, you could subtract 100 from all of player 1's payoffs and this would change nothing whatsoever about the structure of the game.
Also, there is no explicit "winning" or "losing" in your game. You have added that interpretation to your game. And if there was a notion of "winning" or "losing," it won't be because of the payoffs; rather, the payoffs will reflect "winning" or "losing" (e.g. zero-sum games have payoffs where one player's loss is the other's gain. An example is rock, paper, scissors).
So payoffs capture everything about how the players value the outcomes of the game. If you think players value outcomes differently, then this must be baked into the payoffs.