I'm playing Overwatch in ranked, and want to know whether my matchups are "fair". That is I'm not being placed with significantly higher or lower skilled players.
So I can play lots of games (say 30), and record "win" or "lose" for each of them. At the end, I want to know whether or not my wins and losses were random based on a certain level of confidence.
If I'm being matched with players of similar skill, then the outcome of my games should be truly random.
Do you have any suggestions for how I would go about this?
I assume you don't really care about random so much as fair - that is, you want to know whether you're winning roughly as often as you're losing, not whether the sequence of wins and losses is predictable.
$30$ games isn't really enough for a good picture, but we can work with it anyway. Score each game as a $0$ for a loss and a $1$ for a win. Take the average of these scores - presumably, about $0.5$. Now, take the standard deviation: $\sigma = \frac{\sqrt{\sum (x_i - x)^2}}{n - 1}$. That is, the sum of the squares of the differences from the mean, divided by one less than the number of samples. For example, say we have four samples, three wins and a loss. The mean is $(1 + 1 + 1 + 0) / 4 = 3/4$. The standard deviation is $\sqrt{0.25^2 + 0.25^2 + 0.25^2 + 0.75^2}/3$, roughly $0.289$. Finally, take $1.96$ standard deviations on either side of the mean - in this case, that gives an interval $[0.461, 1.039]$. That's pretty close to a $95\%$ confidence interval - the mean is probably between $0.461$ and $1.039$. Since $0.5$ is in that interval, it's reasonable to suppose that the games in this example were fair.
Now, this isn't quite right. There's lots of complicated stuff that goes into confidence intervals; but this approach will work for a first approximation if you're unfamiliar with statistics.