A die is rolled $1000$ times, and the number of times a $1$ appears is $125$. Is it reasonable to assume the die is fair?
If the die was fair, then $P(X=125)= {{1000}\choose{125}}(1/6)^{125}(5/6)^{875}\approx 0.000005$.
However, the expected number of $1$'s is $np=1000*1/6=166.66.....$ How far should the actual number be away from the expectation in order to conclude that the die likely isn't fair?
The standard thing is to ask "How likely is it to get 125 or something further away than that from the expected value, if the die is fair?" In other words, you should calculate the probability $P(X\leq 125)$.
The standard, again, is to say that if this is less than $2.5\%$ likely (since we're only checking one way, and not $X\leq 125$ OR $X\geq 208$, the standard is a $2.5\%$ confidence rather than $5\%$), then the die is probably tampered with, but you can decide yourself where you want that cutoff to be.
For methodical cleanliness, you should really decide on the cutoff you want before you measure / calculate, but since this isn't a statistical analysis that is to go into a peer reviewed article, you can just look at the answer and decide afterwards what conclusion you find reasonable.