Let's say two people play a skill-based game and a winner is decided. Then a rule variation is introduced and the same two people play the game again. This process is then repeated with different pairs of people 100 times. I want to know the effect of the rule variation and whether or not it had a statistically significant impact on the outcome of the game. The only data available is which of the two players won the first game and which of the two won the second game.
What would be the appropriate test to use?
You are comparing a dichotomous variable (win / loss) with the same two players (dependent) in two different settings (no variation / variation). It sounds like McNemar's test would be a good fit for your task.